{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning - CSE-6363\n",
    "**Principal Component Analysis**    \n",
    "**Jesus A. Gonzalez**    \n",
    "**July-26-2019**    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import urllib.request\n",
    "import math\n",
    "from pathlib import Path\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.decomposition import PCA as sklearnPCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 5)\n",
      "   sepal_len  sepal_wid  petal_len  petal_wid        class\n",
      "0        5.1        3.5        1.4        0.2  Iris-setosa\n",
      "1        4.9        3.0        1.4        0.2  Iris-setosa\n",
      "2        4.7        3.2        1.3        0.2  Iris-setosa\n",
      "3        4.6        3.1        1.5        0.2  Iris-setosa\n",
      "4        5.0        3.6        1.4        0.2  Iris-setosa\n",
      "     sepal_len  sepal_wid  petal_len  petal_wid           class\n",
      "145        6.7        3.0        5.2        2.3  Iris-virginica\n",
      "146        6.3        2.5        5.0        1.9  Iris-virginica\n",
      "147        6.5        3.0        5.2        2.0  Iris-virginica\n",
      "148        6.2        3.4        5.4        2.3  Iris-virginica\n",
      "149        5.9        3.0        5.1        1.8  Iris-virginica\n"
     ]
    }
   ],
   "source": [
    "# load the CSV file as a numpy matrix\n",
    "dataset = pd.read_csv(filepath_or_buffer='https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', \n",
    "    header=None, \n",
    "    sep=',')\n",
    "dataset.columns = ['sepal_len', 'sepal_wid', 'petal_len', 'petal_wid', 'class']\n",
    "print(dataset.shape)\n",
    "print(dataset.head())\n",
    "print(dataset.tail())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data into data samples \"X\" and the class labels \"y\"\n",
    "X = dataset.loc[:,'sepal_len':'petal_wid']\n",
    "y = dataset.loc[:,'class']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_len</th>\n",
       "      <th>sepal_wid</th>\n",
       "      <th>petal_len</th>\n",
       "      <th>petal_wid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal_len  sepal_wid  petal_len  petal_wid\n",
       "0        5.1        3.5        1.4        0.2\n",
       "1        4.9        3.0        1.4        0.2\n",
       "2        4.7        3.2        1.3        0.2\n",
       "3        4.6        3.1        1.5        0.2\n",
       "4        5.0        3.6        1.4        0.2"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'], dtype=object)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our dataset is in the form of a matrix. Each row of the matrix is a sample and each column represents a feature. We have 150 training samples and 4 features.\n",
    "    - Feature 1: sepal length\n",
    "    - Feature 2: sepal width\n",
    "    - Feature 3: petal length\n",
    "    - Feature 4: petal width"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Quick Exploratory Analysis\n",
    "Let us look how separable are the classes according to each of the four attributes:    \n",
    "    - Sepal length and sepal width do not make a good work separating the classes, their values overlap\n",
    "    - Petal length and petal width do a better job but not perfect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGoCAYAAABYLN/cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdf1xUVf4/8NcAIsjPCHUtBhc13FEffk2tdMsfPbTQVdNUVNgwxaxMc3Utf7CIFipS6m6yKkJqhT+QNdvss2aP0kwrpcRExbHS1B3QEBTDQWBGuN8/XEZG5sedy50fV1/Pf2LuPfec971zvfPu3nvOUQmCIICIiIhIQbzcHQARERGRo5jAEBERkeIwgSEiIiLFYQJDREREisMEhoiIiBTHx1kVFxQUOKtqInKjXr16uTsEXl+I7lKOXF+clsAAzr3QabVaaDQap9XPttk2227aticlDmKvL+48hs2l1NiVGjeg3NiVGjdwO3ZHry98hERERESKwwSGiIiIFIcJDBERESkOExgiIiJSHFEJTGFhIRISEgDcetkmPj4eCQkJmDJlCsrLy50aIBEREdGd7CYw2dnZSE5ORm1tLQBg6dKlWLhwIXJycvDUU08hOzvb6UESERERNWY3gYmMjERGRobp86pVq0xdterq6tCyZUvnRUdERERkgd1xYGJiYlBcXGz63KZNGwDA0aNHsXnzZmzZssXqtlqtVoYQLaupqXFq/Yd1VVbXGQ1GHNYdAQD0UQc4LQZLnL3fbJtte0LbpGxHKo6gVFcqaduB6oHyBkN3LUkD2e3evRvr1q1DVlYWwsLCrJZz5qA6zh60p0Sw/o9PV6yDOkINANBo2jotBks8bWAztn1vte1JA9kR0b3N4QTm448/xvbt25GTk4PQ0FBnxERERERkk0MJTF1dHZYuXYp27drh1VdfBQA88sgjmDlzplOCIyIiIrJEVAITERGBvLw8AMB3333n1ICIiIiI7OFAdkRERKQ4TGCIiIhIcZjAEJFTNR7Ju6ioCP369UNCQgISEhKwe/dus7I1NTV49dVXER8fj6lTp+Lq1avuCJmIFEBSN2oiIjGys7Oxa9cu+Pv7AwBOnTqFyZMnIzEx0WL5bdu2ITo6Gq+++ir+85//YO3atUhOTnZlyESkELwDQ0ROc+dI3idPnsT+/fvx5z//GUlJSdDr9WblCwoK0K9fPwBA//79cejQIZfGS0TKwTswROQ0d47k3b17d8TGxqJbt25Yt24d1qxZg3nz5pnW6/V6BAUFAQACAgJw/fp1q3WLHSVYySMKKzV2o9EInU4naVutXp79tTWaui2NR1pv4OoR16VQ6rkCSI+dCQwRucxTTz2F4OBg09+pqalm6wMDA1FVdeuHp6qqylTWErEjFLtzNOPmUmrsRyqOQK1WS9pWo5Znf22Npm5L45HWG7h6xHUplHquALdjd3Skbz5CIiKXmTJlCo4fPw4AOHToELp27Wq2vmfPnvjqq68AAAcOHECvXr1cHiMRKQMTGCJymcWLF2PZsmVISEjA0aNH8corrwAAEhMTYTAYEBcXh59//hlxcXHYvn07ZsyY4eaIichT8RESETlV45G8u3btitzc3CZlNm7caPp79erVLouNiJSLd2CIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeKISmAKCwuRkJAAALhw4QLi4uIQHx+PRYsWob6+3qkBEhEREd3JbgKTnZ2N5ORk1NbWAgDS0tIwa9YsbN26FYIgYO/evU4PkoiIiKgxu5M5RkZGIiMjA3PnzgUAFBUV4dFHHwUA9O/fH9988w2eeuopi9tqtVoZQzVXU1Pj1Pp1xVVW1xkNRuiKdQAAreqq02KwpGG/A0sOOryt/sF+srTtDo62faTiiGxtd/Pvppj9vlvaJiKyx24CExMTg+LiYtNnQRCgUqkAAAEBAbh+/brVbTUajQwhWqbVap1af4lQanWdrlgHdYQaAKDRtHVaDJaY9tvrvOMbd27e8XL2MZez7VKd9e/PUX56P8Xst7PbLigocEssRER3cvglXi+v25tUVVUhODhY1oCIiIiI7HE4genSpQvy8/MBAAcOHEDv3r1lD4qIiIjIFocTmHnz5iEjIwPjx4+H0WhETEyMM+IiIiIissruOzAAEBERgby8PABAVFQUNm/e7NSgiIiIiGwRlcAQEUlVWFiIFStWICcnB1qtFqmpqfD29oavry/S09MRHh5uVn7UqFEICgoCcOt/ntLS0twRNhF5OCYwROQ02dnZ2LVrF/z9/QEAS5cuxcKFC6HRaJCbm4vs7GwsWLDAVL5hvKmcnBy3xEtEysGpBIjIaRrGkWqwatUqU9fsuro6tGzZ0qz86dOnUV1djcTEREycOBHHjh1zabxEpBy8A0NETnPnOFJt2rQBABw9ehSbN2/Gli1bzMr7+flhypQpiI2Nxfnz5zF16lTs2bMHPj5NL1ViB9lT8oB8So3daDRCp9NJ2larl2d/bQ1GakvjgUobuHrAUimUeq4A0mNnAkNELrV7926sW7cOWVlZCAsLM1sXFRWF9u3bQ6VSISoqCqGhoSgrK0O7du2a1CN2gD93DgbYXEqN/UjFEajVaknbatTy7K+twUhtaTxQaQNXD1gqhVLPFeB27I4OlMlHSETkMh9//DE2b96MnJwciz9wO3bswPLlywEApaWl0Ov1aN26tavDJCIFYAJDRC5RV1eHpUuXoqqqCq+++ioSEhKwevVqAMDcuXNx8eJFjB07FtevX0dcXBxmz56NZcuWWXx8RETEKwMROVXjcaS+++47i2Xeeust098rV650SVxEpGxMYMij7Nftt7hcV6GTdYJGonuBtX9PYgxUD5QtDiJn4CMkIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4kqYSMBqNmD9/PkpKSuDl5YXU1FR07NhR7tiIiEgOJQWObyNiKoEvTplP7xF+cR8A4ErpMRivnBPVTOX93QEAPdShjsVH9zxJd2C++uor3Lx5E7m5uZg+fTr+8Y9/yB0XERERkVWSEpioqCjU1dWhvr4eer2e090TERGRS0nKPFq1aoWSkhIMHToUFRUVyMzMtFhOq9U2KzhbampqnFq/rrjK6jqjwQhdsQ4AsOl//7WljzpAtrga9juwxH67d9LXO368AksOmv72NRqha/S5QX61+a3i6vu7ONyOPUajETqd4/sshxD/EKeea7Y4+zz31LaJiOyRlMC89957eOKJJzBnzhxcunQJzz//PD755BO0bNnSrJxGo5ElSEu0Wq1T6y8RSq2u0xXroI5Qi65Lo2krR0gAGu2313nHN+4s4Xg1aken00GtbrrfZyuumy94UPyxEcta267gp/dz6rlmi7PPc0fbLiiQ8C4FEZETSEpggoOD0aJFCwBASEgIbt68ibq6OlkDIyIiIrJGUgIzadIkJCUlIT4+HkajEbNnz0arVq3kjo2IiIjIIkkJTEBAAN555x25YyGiu1BhYSFWrFiBnJwcXLhwAfPnz4dKpcJDDz2ERYsWwcvrdl+CmpoavP7667hy5QoCAgKQnp6OsLAwN0ZPRJ6KA9kRkdNkZ2cjOTkZtbW1AIC0tDTMmjULW7duhSAI2Lt3r1n5bdu2ITo6Glu3bsWoUaOwdu1ad4RNRArABIaInCYyMhIZGRmmz0VFRXj00UcBAP3798e3335rVr6goAD9+vUzrT906JDrgiUiReEALkTkNDExMSguLjZ9FgQBKpUKwK1H0devm/dg0+v1CAoKsrq+MbFdvJXcHby5sesqbg094H+l3OFtxbTbZLiJq2UAgPr6elRW/iaqnXLhVmw63PqutXp5vitbQ2HY0niYjAZa1VU5QnKqe/E8ZwJDRC7T+H2XqqoqBAcHm60PDAxEVVWV1fWNie1e7s6u6M3V3NhLdf8bDsLrssPbimn3zuEmwr1a32qu9AKCg0PENXR/OABA/b+pBDRqeb4rW0Nh2GJpmAw5h8JwlrvhPHd0mAY+QiIil+nSpQvy8/MBAAcOHEDv3r3N1vfs2RNfffWVaX2vXr1cHiMRKQMTGCJymXnz5iEjIwPjx4+H0WhETEwMACAxMREGgwFxcXH4+eefERcXh+3bt2PGjBlujpiIPBUfIRGRU0VERCAvLw/ArXnUNm/e3KTMxo0bTX+vXr3aZbERkXLxDgwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeJIno16/fr12LdvH4xGI+Li4hAbGytnXERERERWSUpg8vPz8cMPP2Dbtm2orq7Gxo0b5Y6LiIiIyCpJCczXX3+N6OhoTJ8+HXq9HnPnzpU7LiIiIiKrJCUwFRUVuHjxIjIzM1FcXIxp06Zhz549UKlUZuW0Wq0sQVpSU1Njsf7Duiq72/ZRB9gtoyu2Xo/RYISuWGe3jgZa1VWb6wNLDoquy9dohM6B8o3p6x3/PgJLbu+nwWiETtd0v8ury80+V9eLPzZiGa207Qoh/iFOPZdtsXaeH6k4Ilsbve/r7VDbRM60X7df8rYD1QNli4M8n6QEJjQ0FB06dICvry86dOiAli1b4urVq7j//vvNymk0GlmCtESr1Vqsv0QotbutRtPWbhlb9eiKdVBHqO3WIbo9r/Oi69LpdFCrxbdtprOE76NRbNbaPltx3XzBgxLjs6FZ+91Mfno/p57Ltlg7z0t19s9zsTRqy/tmqe2CggLZ2iUiag5JCUyvXr3wwQcfYPLkybh8+TKqq6sRGhoqd2xEdBfauXMnPvroIwBAbW0ttFotvvnmGwQHBwMAlixZgqNHjyIg4Nad0rVr1yIoKMht8RKRZ5KUwDz55JP4/vvvMXbsWAiCgJSUFHh7e8sdGxHdhUaPHo3Ro0cDAN544w2MGTPGlLwAQFFREd59912EhYW5K0QiUgDJ3aj54i4RNceJEydw5swZLFq0yLSsvr4eFy5cQEpKCsrLyzF27FiMHTvWjVESkaeSnMAQETXH+vXrMX36dLNlN27cwHPPPYfJkyejrq4OEydORLdu3fCHP/yhyfZiXzBW8svIzY1dV3HrxXf/K+V2SjYlpt0mnR2ulgG4lYhWVv4mqp1y4VZsOly3U9I+rf52zLY6YthiqZOGvY4YnuBePM+ZwBCRy1VWVuKXX35Bnz59zJb7+/tj4sSJ8Pf3BwD06dMHp0+ftpjAiH2x2tqL0ErQ3NhNL3t7XXZ4WzHt3tnZIdyr9a3mSi8gODhEXEP3hwMA1Ormv0fZ+IV0MR06LLHUSUNMxw93uxvOc0c7CXAqASJyue+//x5//OMfmyw/f/484uPjUVdXB6PRiKNHj6Jr165uiJCIPB3vwBCRy507dw4RERGmz5s2bUJkZCQGDRqEESNGYNy4cWjRogVGjhyJhx56yI2REpGnYgJDRC73wgsvmH2ePHmy6e+pU6di6tSprg6JiBSGCQwRETX146d2i4RfvCZbc8d0za/r5nX5Bngkz8d3YIiIiEhxmMAQERGR4jCBISIiIsW5J9+B+eKU5z0nFfv890z1uaaTJ4p1+JS07f6nvLpcXNslEib8e7CX49u4qJ0jFUdkmzyRs+USEcmDd2CIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREisMEhoiIiBTnnuyFREQkOxEj1zbReaj8cZDHk7sn7OAunj9btjPwDgwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcZqVwFy5cgUDBgzA2bNn5YqHiIiIyC7JCYzRaERKSgr8/PzkjIeIiIjILsndqNPT0zFhwgRkZWVZLaPVakXVdVhXZbdMH3WA2eeamhqL9euK7dclhvrqt1bXtbtZB1z1btp22B8tlt9UrLPT1s+i4zL63UR5Wbno8nIy1jmv7ep628fIaDRCpzMv43/F8VjstSO2bam0enH/JhpYPc8r5IkHsB6TtbaJiDyBpARm586dCAsLQ79+/WwmMBqNRlR9JYL9PvEajXk/d61Wa7F+MXWJEe7V2uq68vIyhIdbWP+AWva27nSt+hLCW4dLaqe5ysvKndf2g7aPnU6ng1p9Rxmvy7K3I7ptiTRqcf8mGlg7z+WaHRuwHpOltgsKJMwAbsGoUaMQFBQEAIiIiEBaWpppXV5eHnJzc+Hj44Np06bhySeflKVNIrq7SEpgPvzwQ6hUKhw6dAharRbz5s3DunXr0Lq1+B9iIro31dbWAgBycnKarCsrK0NOTg4+/PBD1NbWIj4+Ho8//jh8fX1dHSYReThJCcyWLVtMfyckJGDx4sVMXohIlNOnT6O6uhqJiYm4efMm/vrXv6JHjx4AgOPHj+Phhx+Gr68vfH19ERkZidOnT6N79+5ujpqIPA2nEiAil/Lz88OUKVMQGxuL8+fPY+rUqdizZw98fHyg1+tNj5YAICAgAHq93mI9Yt/PcdW7PIEljr+XpK+3HVdzY294V0rK+2I6vf39KS+vsbi8vr4elZW/iWqnXJDvvToxMdtjNBihu+O9RXvvMbqbVnVV0e+sSY292QmMpdvARETWREVFoX379lCpVIiKikJoaCjKysrQrl07BAYGoqrq9ov4VVVVZglNY2LfsbP2HpHsvM47vk1n23E1N3bTu1IS3hdT32f/va8ruGZxuVfpBQQHh4hr6H753qtThzb/XTVdsQ7qCHneeXMVjaat685zJ2iI3dF37DiQHRG51I4dO7B8+XIAQGlpKfR6vekRdPfu3VFQUIDa2lpcv34dZ8+eRXR0tDvDJSIPxUdIRORSY8eOxYIFCxAXFweVSoVly5YhJycHkZGRGDRoEBISEhAfHw9BEDB79my0bNnS3SETkQdiAkNELuXr64uVK1eaLevZs6fp73HjxmHcuHGuDouIFIaPkIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFIe9kO5Cv5TZn5G7Q+sAu2VcqsT2AEb+V8qlTd7oYDuWtWl+u3TP219xqunCwxaWNVJeVo7S38Ix8L4utxd2HipzZETKxDswREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxJE0lYDQakZSUhJKSEhgMBkybNg2DBg2SOzYiIrrTj5+KL2tp+gIPE3zluMPbVN7f3QmRkNJISmB27dqF0NBQvP3226ioqMCzzz7LBIaIiIhcRlICM2TIEMTExJg+e3t7yxYQEd3d7N3B3bRpE3bs2IGwsDAAwBtvvIEOHTq4K1wi8lCSEpiAgFszGev1esycOROzZs2yWE6r1YqqT1dsf/Zkreqq2eeamhqL9YupS5SrZVZX3bxZh/Lyput19TrZ27rTf401KK68KK2dRspR7fA2xrqbKC8rb3bbUri17eD7oNNJ/G7voNWL+zfRwOp5XiFPPID1mKy13Vz27uAWFRUhPT0d3bp1k71tIrp7SEpgAODSpUuYPn064uPjMWLECItlNBqNqLpKhFK7ZTSatmaftVqtxfrF1CVGuFdrq+vKy8sQHm5h/QNq2du6k1fpBQQHh0hqx6zN1gEOb1NeVo7w1uHNblsKd7ZdXd8CarW07/ZOGrW4fxMNrJ3npTp5znPAekyW2i4oKGh2e/bu4BYVFSErKwtlZWUYOHAgXnrppWa3SUR3H0kJTHl5ORITE5GSkoK+ffvKHRMR3cXs3cEdNmwY4uPjERgYiBkzZuDLL7/Ek08+2aQesXeHnHUn6U6BJbbvipVXO34HseHOo04v7Y6blDYbiGmzvLzG4vL6+npUVv4muW277QqW90vqcWrMaDBCVyzfHU5X0Kquuuw8dwapsUtKYDIzM1FZWYm1a9di7dq1AIDs7Gz4+flJqY6I7jHW7uAKgoDnn38eQUFBAIABAwbg1KlTFhMYsXd4rd3Fkp3XeZurz1Zcd7jKhjuP6vuk3QGU0mYDMW1ewTWLy+W6U2zV/ZbvxqpDm3+nVFesgzpCnjuurqLRtHXdee4EDbE7eodXUgKTnJyM5ORkKZsS0T3O1h1cvV6P4cOHY/fu3WjVqhXy8/MxZswYN0VKRJ5M8jswRERSWLqDGxsbi+rqaowfPx6zZ8/GxIkT4evri759+2LAgAFujpiIPBETGCJyKXt3cEeNGoVRo0a5MCIiUiImMEREd3JktFsX2O+GEXXFtPlLtUzDVjjI2ui9/7Uzqq+tEXy7hUrvkHLy2iHJ2zan3QZfnCqFrrhKtl64chrcpa39QhJxLiQiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKo5hu1F+cMu8e5uwuY0erf7a6rrLuNwRXWxhC+6zlbXr6PyRXWKRw+3X7HSqvq9DJOnEjEdHdgndgiIiISHGYwBAREZHiKOYREhGRFIElB+3OFC2WqNFpy+Qbnbay0oDw1rJVJ2tsd5uG0XTLa8rx27ViN0dDYvAODBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKIymBqa+vR0pKCsaPH4+EhARcuHBB7riI6C5l7/qRl5eH0aNHY9y4cfjyyy/dFCUReTpJCcwXX3wBg8GA7du3Y86cOVi+fLnccRHRXcrW9aOsrAw5OTnIzc3Fhg0bsGrVKhgMBjdGS0SeSlICU1BQgH79+gEAevTogZMnT8oaFBHdvWxdP44fP46HH34Yvr6+CAoKQmRkJE6fPu2uUInIg0maSkCv1yMwMND02dvbGzdv3oSPj3l1BQUFouq7T0IM992vAqqdN9zzfQ8Ms77yAcfqqmteKGb63RctY22OCXJwv++WtrsAwGU3td2yi9PbLrhs/d+p2H/DjrB1/dDr9QgKCjKtCwgIgF6vb15s9/XCZctVOCyoRRu7Zf6fnOeqzOe9rLHZ4sZ/r832oNq17cn0O+bs30SpCgrExSTlWiMpgQkMDERV1e05Nerr65skL7169ZJSNRHd5WxdP+5cV1VVZZbQNOD1hYgkPULq2bMnDhw4AAA4duwYoqPdd1eAiJTF1vWje/fuKCgoQG1tLa5fv46zZ8/y+kJEFqkEQRAc3ai+vh6LFy/GTz/9BEEQsGzZMnTs2NEZ8RHRXcbS9ePAgQOIjIzEoEGDkJeXh+3bt0MQBLz00kuIiYlxd8hE5IEkJTCuduXKFYwePRobN240S5Q2bdqEHTt2ICwsDADwxhtvoEOHDrK1O2rUKNPt64iICKSlpZnW5eXlITc3Fz4+Ppg2bRqefPJJ2dq11/aSJUtw9OhRBAQEAADWrl1r8Ta7VOvXr8e+fftgNBoRFxeH2NhY07p9+/ZhzZo18PHxwZgxYzBu3DjZ2rXXtrO/7507d+Kjjz4CANTW1kKr1eKbb75BcHAwAOd+5/baduZ3bjQaMX/+fJSUlMDLywupqalm/86c/Z3LwWg0IikpCSUlJTAYDJg2bRoGDRpkWu/J+2Avdmef981RV1eH5ORknDt3Dt7e3khLS0NkZKRpvaced3txe/Ixb2Dtd9FTj3kDWX/PBQ9nMBiEV155RXj66aeFM2fOmK2bM2eOcOLECae0W1NTI4wcOdLiusuXLwvDhw8XamtrhcrKStPfrmhbEARhwoQJwpUrV2Rrr7HDhw8LL730klBXVyfo9Xph9erVpnUGg0EYPHiwcO3aNaG2tlYYPXq0cPnyZZe0LQjO/b7vtHjxYiE3N9f02dnfua22BcG53/nnn38uzJw5UxAEQfj666+FGTNmmNY5+zuXy44dO4QlS5YIgiAIV69eFQYMGGBa5+n7YCt2QXDtee+ozz//XJg/f74gCLf+/b788sumdZ583G3FLQiefcwFwfrvoicfc0GQ//fc40fiTU9Px4QJE9CmTdO3/4uKipCVlYW4uDisX79e1nZPnz6N6upqJCYmYuLEiTh27JhpnbO7etpqu76+HhcuXEBKSgomTJiAHTt2yNYuAHz99deIjo7G9OnT8fLLL2PgwIGmdWfPnkVkZCRCQkLg6+uLXr164ciRIy5pG3Du993YiRMncObMGYwfP960zFXdey217ezvPCoqCnV1daivr4derzd7Id/Z37lchgwZgr/85S+mz97e3qa/PX0fbMUOuO68l2Lw4MFITU0FAFy8eBHh4eGmdZ583G3FDXj2MQes/y568jEH5P89l9QLyVV27tyJsLAw9OvXD1lZWU3WDxs2DPHx8QgMDMSMGTPw5ZdfynZb38/PD1OmTEFsbCzOnz+PqVOnYs+ePQ539ZS77Rs3buC5557D5MmTUVdXh4kTJ6Jbt274wx/+IEvbFRUVuHjxIjIzM1FcXIxp06Zhz549UKlUTt9vW20Dzv2+G1u/fj2mT59utszZ+26rbWd/561atUJJSQmGDh2KiooKZGZmmta5ar+bq+HRml6vx8yZMzFr1izTOk/fB1uxA64776Xy8fHBvHnz8Pnnn2P16tWm5Z5+3K3FDXj2Mbf1u+jJx9wZv+cefQfmww8/xLfffouEhARotVrMmzcPZWVlAABBEPD8888jLCwMvr6+GDBgAE6dOiVb21FRUXjmmWegUqkQFRWF0NBQU9tiu3o6o21/f39MnDgR/v7+CAwMRJ8+fWS9ExAaGoonnngCvr6+6NChA1q2bImrV68CcP5+22rb2d93g8rKSvzyyy/o06eP2XJn77uttp39nb/33nt44okn8Nlnn+Hjjz/G/PnzUVtbC8A1+y2XS5cuYeLEiRg5ciRGjBhhWq6EfbAWu6vO++ZKT0/HZ599hoULF+LGjRsAlHHcLcXt6cfc1u+iJx9zZ/yee3QCs2XLFmzevBk5OTnQaDRIT09H69atAdzKNIcPH46qqioIgoD8/Hx069ZNtrZ37NhhGuK8tLQUer3e1Lazu3raavv8+WRmDCMAACAASURBVPOIj49HXV0djEYjjh49iq5du8rWdq9evXDw4EEIgoDS0lJUV1cjNDQUANCxY0dcuHAB165dg8FgwJEjR/Dwww+7pG1nf98Nvv/+e/zxj39sstwV3Xutte3s7zw4ONh0kQsJCcHNmzdRV3dr+EVnf+dyKS8vR2JiIl5//XWMHTvWbJ2n74Ot2F113kv173//23S739/fHyqVyvQIzJOPu624Pf2Y2/pd9ORj7ozfc0X0QgKAhIQELF68GKdOncKNGzcwfvx4/Pvf/0ZOTg58fX3Rt29fzJw5U7b2DAYDFixYgIsXL0KlUuG1115DYWGhS7p62ms7Ozsbe/bsQYsWLTBy5EjExcXJ1jYAvPXWW8jPz4cgCJg9ezauXbtmOuYNb7gLgoAxY8bgz3/+s8vadub33eDdd9+Fj48PJk2aBODWm/Gu6t5rq21nfudVVVVISkpCWVkZjEYjJk6cCAAu+87lsGTJEnz66admvRZiY2NRXV3t8ftgL3ZXnPdS3bhxAwsWLEB5eTlu3ryJqVOnorq62uPPHXtxe/Ixb8zS76KnHvPG5Po9V0wCQ0RERNTAox8hEREREVnCBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOExgCPPnz8eBAwfsLpPq4sWL2LdvH4Bb3efOnj1rtWxGRgZiYmLwySefSGrrb3/7G3r37m2zDSJyDzHXlbKyMixevLjJ8hUrVmDnzp0AgM2bNwO4NbrrihUrrNZVXFyMnj17Yu7cuZLi3bNnD4YMGWKzDXIfJjDkdIcPH8bRo0dFl580aZLZaKSOWLp0KTQajaRticj9WrdubTGBaWzdunWi6+vUqRPeeustSbEMGTIEL774oqRtyfk8ei4kMnfu3DksWLAAPj4+8Pb2xltvvYW2bdti5cqV+P777yEIAiZNmoShQ4ciISEBUVFROHfuHARBwN///neEhYUhJSUFv/76KyoqKtC/f/8m867cyWg0YtGiRbhw4QLq6+sxa9YsPPbYYxgxYgQeffRR/Pjjj1CpVFi7di0CAwPxxhtv4OTJkwgPD0dJSQnWrFmDrKws1NTUmEaEXLNmDcrLy1FdXY1Vq1ZBrVZbbLuwsBBLly6FIAho27YtVqxYgalTp6Jz5874+eef0apVK/Tu3Rtff/01KisrsXHjRoSEhMh+3InuNc6+1jz77LN49913ERwcjMceewybN29Gly5d8Oyzz2LlypWYP38+8vLy8Nlnn2HdunUICwuD0WhEhw4dsG7dOvz2229YvHgxunfvjsLCQiQmJuLq1auIi4szmwi1sZqaGtMAoUajEQsXLsS5c+fw5ZdfoqamBmVlZZg4cSL27t2Ln3/+GXPnzsXgwYNddchJAt6BUZBvv/0WXbt2xaZNm/Dyyy/jt99+w1dffYXi4mLk5ubigw8+QGZmJiorKwEAPXv2RE5ODoYOHYr169fj0qVL6NGjBzZs2IBt27Zh27Ztdtv817/+hfvuuw9btmzB2rVr8eabbwK4NXrrsGHDsHnzZrRp0wYHDhzA3r17ce3aNezYsQPLli3DpUuX4OXlhRdffBHDhw/HoEGDAAADBgzABx98gP79+2PPnj1W2164cCHS0tLwr3/9C3379jU9FurevTvef/99GAwG+Pn5YdOmTejUqRO+//775h5iIoLzrzWDBg3CwYMHUVBQgIiICHzzzTc4c+YMfv/738PX19dU7u2338amTZuwYcMG+Pn5AQCmTZuGkJAQ010aHx8fbNiwAf/85z/x/vvvW92n3NxcPPjgg9i+fTuWL1+OwsJCALeuZdnZ2Zg6dSq2bduGf/7zn3jzzTdNj6vIc/EOjIKMHTsW2dnZeOGFFxAUFITZs2fjp59+QlFRERISEgAAN2/exMWLFwHANClgz549sW/fPoSGhuLEiRM4fPgwAgMDYTAY7Lb5008/oaCgAMePHzfVX1FRAQDo0qULAKBdu3aora1FSUkJevToAQAICwszGxq9sYY5LsLDw1FeXm617StXrqBjx44AYDYcdsM8QMHBwejUqZPp74YJCImoeZx9rXn66aeRmZmJdu3aYfbs2cjJyYEgCHj66adNZcrLyxEYGIj77rsPAKzO6dOlSxeoVCq0bt0aNTU1Vvfpl19+Qf/+/QEA0dHRiI6Oxs6dO02PnIOCgtCxY0eoVCqEhITweqIAvAOjIHv37kWvXr3w/vvvY8iQIXj33XfRoUMHPPbYY8jJycH777+PoUOHIiIiAgBw8uRJAMDRo0fRqVMn7Ny5E0FBQVi5ciUSExNRU1MDezNJdOjQAcOGDUNOTg6ys7MxZMgQ02MalUplVvahhx7CsWPHAAC//fYbzp8/DwDw8vJCfX29w/vbpk0bUx1ZWVn4/PPPHa6DiBzn7GtNdHQ0iouLcfz4cQwYMAA3btzA3r17TQkGcGt2+uvXr5tmpD9x4oRpXeO67rwOWdOxY0dTHTqdDnPmzHFoe/I8vAOjIN26dcPrr7+OjIwMeHl5YcGCBejSpQu+++47xMfH48aNGxg8eDACAwMBAB999BHee+89+Pv746233kJ5eTn++te/oqCgAP7+/mjfvj0uX75ss80JEyYgOTkZzz33HPR6PeLj4+HlZTnvHThwIA4cOIAJEyYgPDwcfn5+aNGiBaKjo7Fu3TqHZ1B+4403kJSUBC8vL7Ru3RqTJk3CBx984FAdROQ4V1xrHnnkERQXF8PLywuPPPIIzpw5g4CAANMdXh8fH6SlpWHKlCkICQmBj8/tn6uOHTvitddeszh7uzUTJkxAUlISnnvuOdTV1SEpKQk///yzDEeL3IWTOd6lGmb7bHgE4wpnz57F6dOnMWzYMFRUVGD48OH48ssvzZ5p25ORkYHw8PBmzbbsjn0nuld58r+34uJi/PWvf0VeXp7kOnbu3IlffvkFr732moyRkRz4CIlk065dO/zf//0fxo0bhxdeeAGvvfaaQ8lLg/fee69Z48BotVpJ2xLR3efMmTPNGgcmKytL5ohILrwDQ0RERIrDOzBERESkOExgiIiISHGYwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOE6bzLGgoMBZVRORG/Xq1cvdIfD6QnSXcuT64tTZqMUGotVqodFonBmKJIzLMYzLMUqMy5MSh169ennsMbRHqXEDjN0dlBo34Fjsjl5f+AiJiIiIFIcJDBERESkOExgiIiJSHCYwREREpDiiEpjCwkIkJCSYLfvkk08wfvx4pwRFREREZIvdXkjZ2dnYtWsX/P39Tcu0Wi127NgBQRCcGhwREZESfXGqtFnbD+7SVqZI7l52E5jIyEhkZGRg7ty5AICKigqsWLECSUlJWLhwoc1ttVqtqCBqampEl7XmsK7K7HMfdUCz6gPkicsZGJdjGJdjPDUuZ7L0Y8MfELqXXLx4EQ888IC7w3CI3QQmJiYGxcXFAIC6ujr87W9/Q1JSElq2bGm3crF9v+Xo414imF+ANJrmX3w8te8943IM43KMUsaBIbpX5efno7CwEC+++CIAIDMzExMmTEBoaKjkOlNSUvDuu+/KFaJLODSQXVFRES5cuIDFixejtrYWZ86cwdKlS/G3v/3NWfERERGRBc899xy6dOmCyspKGI1GpKamokWLFigrK0N6ejp8fG79xP/3v/9FRkYG/P390blzZ4wcORIrV66Et7c3DAYDJk2ahPPnz+OTTz5BWFgYdu7cCV9fX/Tp0wcPP/yw2bZDhw5FWloaAgICUFpaitWrV6NFixZu2X+HeiF1794d//nPf5CTk4NVq1ahU6dOTF6IiIjcoEePHkhKSjJ91ul0+N3vfoc///nPUKlUpuXXrl2DXq9Hnz598Pjjj2PXrl24du0a/Pz8UFVVhRs3bqB9+/YYMWIE3n//faSnpyMtLQ07d+5ssq2XlxfGjh2Lxx9/HJcvX8bly5fdsesA2I2aiIhIkYKDg01/37x5EzNmzEC3bt2QlZWFoqIizJ49G7Nnz8b999+POXPmAABSU1MhCAIef/xxvPbaa/jTn/6Etm1vv3JRX19vlvzcuW1+fj4+/fRTtG3bFg888IBbO/OIeoQUERGBvLw8u8uIiIjI9S+B+/j4IDc3Fy1btkRQUBDUajX+/ve/AwDOnj2LlStX4sEHH8TDDz+MZ555BklJSTh58iRqamowcOBAqNVqbNiwAc8//zySkpIQEBCA2NhY1NTUmG0bGhqKS5cuYffu3fj1119x7do1REREuHRfTfvsllaJiIhIksceewyPPfaY6fPy5csBAMuWLbNYvmPHjli3bp3ZsoyMDLPPixcvNv3dr18/s3V3btu4bXfiIyQiIiJSHCYwREREpDhMYIiIiEhx+A4MERGR3H78tHnbdx4qTxx3Md6BISIiUpD8/HxkZWWZPmdmZuLatWtOaevUqVPYtWuX3XLFxcVISUlxSgzW8A4MERGRAokdiXfTpk146KGH8MQTT2DGjBlIS0vDqlWrTCPxJicnY+LEiejQoQMmTpyIDRs2mEbe7dSpE3799Vf8+OOPeO+991BXV4fHH38c0dHRyMrKQlBQENq3b4+YmBgAwK+//orly5cjLCwMLVu2xDPPPIOYmBj07t0bM2fONBtzprl4B4aIiEiBxI7EO3LkSPznP/+BTqfDAw88gE8++cRsJN6ffvoJ9fX1WLZsGQwGg9nIuw2ysrKQlJSE9PR0dOrUCVlZWVi0aBHefPNNHD16FFVVtyZUzsnJwYsvvoiUlBQYjUb897//RUhICJYuXSpr8gIwgSEiIlIksSPxBgYGAgC2bt2KMWPGWByJNygoCEDTkXcbGI1GU1J04cIFsxF7VSqVaUTe+vp6eHl5mS1vqFtufIRERG5jNBqRlJSEkpISGAwGTJs2Db/73e/w8ssv4/e//z0AIC4uDn/605/cGyiRo1z8Eq6tkXgB4JlnnsE777yDefPm4YEHHmgyEm+DO0febfDCCy/gzTffBHBroLsXX3wRqampCAsLQ+/evU1J0nPPPYdVq1ahTZs2aNWqFdq3b++8fXZazUREduzatQuhoaF4++23UVFRgWeffRbTp0/H5MmTkZiY6O7wiDySoyPxAkDfvn3Rt29fAEBQUFCTkXg3bNgAwPKovQ1tvfXWW2bLV6xYYfa5IcFZuXKlaZlWqzXVLTcmMETkNkOGDDG9/AcA3t7eOHnyJM6dO4e9e/eiffv2SEpKMv3fHRFRAyYwROQ2AQEBAAC9Xo+ZM2di1qxZMBgMiI2NRbdu3bBu3TqsWbMG8+bNa7KtVqtFTU0NtFpts+PQFVc1rV91tdn1WiNX3O7A2F1PqXEDzo2dCQwRudWlS5cwffp0xMfHY8SIEaisrDS9nPjUU0+ZvUjYmEajgVarhUajaXYMJUKphfqdN5uwXHG7A2N3PaXGDTgWe0FBgUN1M4EhIrcpLy9HYmIiUlJSTM/np0yZgoULF6J79+44dOgQunbt6uYoiRy3X7e/WdsPVA+UJY67magEprCwECtWrEBOTg60Wi1SU1Ph7e0NX19fpKenIzw83NlxEtFdKDMzE5WVlVi7di3Wrl0LAJg/fz6WLVuGFi1aIDw83OodGCK6t9lNYLKzs7Fr1y74+/sDAJYuXYqFCxdCo9EgNzcX2dnZWLBggdMDJaK7T3JyMpKTk5ssz83NdUM0RMqQn5+PwsJCvPjiiwBu/Y/AhAkTEBoaKmr7U6dO4cyZM3jmmWeslvnqq6/g6+trujPqyLauYjeBiYyMREZGBubOnQsApv7dAFBXV4eWLVs6N0IiIiJqQupUAuPHj8evv/6KjIwMnDt3Dr1794bBYMAvv/wCg8EAX19f9OjRAy1btsT8+fPxu9/9Dnq9HlFRUVanFlCr1cjNzYWPjw9CQkIsvngvN7sJTExMDIqLi02fG5KXo0ePYvPmzdiyZYvVbcW+eSz1LeXDuqY9B0xty9CDwFPf/GZc4h2pOAKj0YgjFUeslul9X28XRnSbJx4vwHPjIiJzPXr0wGuvvYb58+cDuDWVwB//+Ec8/fTTTaYSePvtt9G+fXs88MAD8PX1Na0bNWoUHnvsMcydOxfvvPMOfvjhB3z00Udm7YwePRoRERF44YUX0KlTJwC3phZYvHgxAgMDcerUKQQGBmLs2LH49ddfsXHjRhfsvcSXeHfv3o1169YhKysLYWFhVsuJffNY6hvWlnoO3G67+T0IPPXNb8YlXqmuFDqdDmq12moZjdo9MXvi8QJsx+VoLwEich5LUwkYDAZkZWXhlVdewaZNmwAA6enpAG5PJdB45urg4GAYjUbTVADe3t5N2vH394eXl5dZUnTn1AIFBQV46KGH0Lt3b5c9mXE4gfn444+xfft25OTkiH7eRkREdC9xdS8iR6YSyM/PN9s2MDAQXbt2xZIlS1BRUSGqY86dUwu0a9cOP/zwA3788UfU1tairq7OYjIkJ4cSmLq6OixduhTt2rXDq6++CgB45JFHMHPmTKcER0REROaaO5XAndsDQHV1NVq0aAE/Pz+MGTMG0dHRAIBhw4aZyjRMCWBtagFXE5XAREREIC8vDwDw3XffOTUgIiIicq1Zs2a5OwSHebk7ACIiIiJHMYEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOExgiIiISHGYwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOExgiIiISHF8xBQqLCzEihUrkJOTgwsXLmD+/PlQqVR46KGHsGjRInh5MQ8iIscZjUYkJSWhpKQEBoMB06ZNQ6dOnXiNISK77F4VsrOzkZycjNraWgBAWloaZs2aha1bt0IQBOzdu9fpQRLR3WnXrl0IDQ3F1q1bkZ2djdTUVF5jiEgUuwlMZGQkMjIyTJ+Liorw6KOPAgD69++Pb7/91nnREdFdbciQIfjLX/5i+uzt7c1rDBGJYvcRUkxMDIqLi02fBUGASqUCAAQEBOD69etWt9VqtaKCqKmpEV22MV1xlfW2VVcdru9OUuNytrs9rsCSgzbX6x/sJ7ouXYUORqMROp3Oahmt3j3H8m7/HsUICAgAAOj1esycOROzZs1Cenq6qGuMVquVLVZL1xI5riHWeOp3LwZjdz2lxg04N3ZR78A01vhZdFVVFYKDg62W1Wg0ourUarWiyzZWIpTaaLutw/XdSWpcznbXx+V13vb6zuLbKNWVQqfTQa1WWy2jUbvnWCrxeywoKJC9vUuXLmH69OmIj4/HiBEj8Pbbb5vW2brGaDQa2Y6hpWuJHNcQazz1uxeDsbueUuMGHIvd0euLw2/GdenSBfn5+QCAAwcOoHfv3o5WQUQEACgvL0diYiJef/11jB07FgCvMUQkjsMJzLx585CRkYHx48fDaDQiJibGGXER0T0gMzMTlZWVWLt2LRISEpCQkIBZs2bxGkNEdol6hBQREYG8vDwAQFRUFDZv3uzUoIjo3pCcnIzk5OQmy3mNISJ7OLgCERERKQ4TGCIiIlIcJjBERESkOExgiIiISHGYwBAREZHiMIEhIiIixWECQ0RERIrj8FQCRPei/br9dssMVA90ehxERHQL78AQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxJA1kZzQaMX/+fJSUlMDLywupqano2LGj3LERERERWSTpDsxXX32FmzdvIjc3F9OnT8c//vEPueMiIiIiskrSHZioqCjU1dWhvr4eer0ePj6Wq9FqtaLqq6mpEV22MV1xldV1WtVVh+u7k9S4nM3dcQWWHLS43NdohK7kIPQP9mtm/Tqb6/X14vddV6GD0WiETme9Tq3efn26CtsxAUCOLsdumd739Tb97e7v0RpPjYuIqDFJCUyrVq1QUlKCoUOHoqKiApmZmRbLaTQaUfVptVrRZRsrEUqtrtNo2jpc352kxuVsbo/L67zFxTqdDmq1GujczNis1G/iQP2lutLbcVmhUduvr1Rn/VxzROO23P49WmErroKCAhdHQ0RkmaRHSO+99x6eeOIJfPbZZ/j4448xf/581NbWyh0bERERkUWS7sAEBwejRYsWAICQkBDcvHkTdXV1sgZGREREZI2kOzCTJk1CUVER4uPj8fzzz2P27Nlo1aqV3LER0T2gsLAQCQkJAICioiL069cPCQkJSEhIwO7du90cHRF5Kkl3YAICAvDOO+/IHQsR3WOys7Oxa9cu+Pv7AwBOnTqFyZMnIzEx0c2REZGnk5TAEBHJITIyEhkZGZg7dy4A4OTJkzh37hz27t2L9u3bIykpCYGBgRa31Wq1svWYstSjUY6ejNYouacXY3c9pcYNODd2JjBE5DYxMTEoLi42fe7evTtiY2PRrVs3rFu3DmvWrMG8efMsbqvRaGTryWWpR6McPRmt8dQeaGIwdtdTatyAY7E72suRUwkQkcd46qmn0K1bN9Pfp06dcnNEROSpmMAQkceYMmUKjh8/DgA4dOgQunbt6uaIiMhT8RESEXmMxYsXIzU1FS1atEB4eDhSU1PdHRIReSgmMETkVhEREcjLywMAdO3aFbm5uW6OiIiUQFEJzBenxA/n3rjs4C7OexmPLPjxU9vrOw/17PqJiMjj8R0YIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFUVQvJCIiukuxdyE5iHdgiIiISHGYwBAREZHiSH6EtH79euzbtw9GoxFxcXGIjY2VMy4iIiIiqyQlMPn5+fjhhx+wbds2VFdXY+PGjXLHRURERGSVpATm66+/RnR0NKZPnw69Xo+5c+fKHRcRERGRVZISmIqKCly8eBGZmZkoLi7GtGnTsGfPHqhUKrNyWq1WVH01NTWiyuqKq6SEC63qqqTtxMblau6OK7BEZ3G5wWiETmd5nRldlswRmdPX3z42ugodjHbi0upFnHsVIvZLhMZtuft7tMZT4yIiakxSAhMaGooOHTrA19cXHTp0QMuWLXH16lXcf//9ZuU0Go2o+rRaraiyJYL4yRzN45A2maPYuFzN7XF5nbe4WKfTQa1WuzYWSzrfPjalulK7cWnU9o9lqU7auWerLbd/j1bYiqugoMDF0RARWSapF1KvXr1w8OBBCIKA0tJSVFdXIzQ0VO7YiIiIiCySdAfmySefxPfff4+xY8dCEASkpKTA29tb7tiIiIiILJLcjZov7hIREZG7cCA7IiIiUhzOhURERCTRft1+UeUGqgc6NY57Ee/AEBERkeIwgSEiIiLFYQJDRG5VWFiIhIQEAMCFCxcQFxeH+Ph4LFq0CPX19W6Ojog8FRMYInKb7OxsJCcno7a2FgCQlpaGWbNmYevWrRAEAXv37nVzhETkqZjAEJHbREZGIiMjw/S5qKgIjz76KACgf//++Pbbb90VGhF5OPZCIsXaX3HK8gqdv2sDcUDjHgu6Cp3VKQrulR4LMTExKC4uNn0WBME0p1pAQACuX79udVutVivbvE2W5lmTOoeaGEqeb8pZsVubY61B4znOHK/7IADA12iE7n9/m+p9sJ/kegHx86SJmXPNGp4vljGBISKP4eV1+6ZwVVUVgoODrZbVaDSyzSdlaZ41qXOoieGp82CJ4bTYrcyxZtK5GW3+r26L86I1p16InydNzJxr1twr54ujc63xERIReYwuXbogPz8fAHDgwAH07t3bzRERkadiAkNEHmPevHnIyMjA+PHjYTQaERMT4+6QiMhD8RESEblVREQE8vLyAABRUVHYvHmzmyMiIiXgHRgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jQrgbly5QoGDBiAs2fPyhUPERERkV2SExij0YiUlBT4+fnJGQ8RERGRXZITmPT0dEyYMAFt2rSRMx4iIiIiuySNA7Nz506EhYWhX79+yMrKslpO7PwHYudKsDRfiRibim/PVdFHHSB6O0+df8LdcVmbs8RgNEKnEzcviBzKq8stryj7zOyjT91NlF+5PW9S9f1dzNbn6HJkj00Mo43j1Zx5U5rL3ecXEblH47na7PGE+dokJTAffvghVCoVDh06BK1Wi3nz5mHdunVo3bq1WTmx8x+InSvB0nwljnJkfhNPnX/C7XFZmbPE4jwjTnS2wvpEf42Vl5UjvHX47QUPui5GW2wdr+bMm9Jcts4vR+cqISJyFkkJzJYtW0x/JyQkYPHixU2SFyIiIiJnYTdqIiIiUpxmz4WUk+Oe9weIiIjo3sU7MERERKQ4nI2aiMiCL0417TQwuIv9TgBSt1OKwJKDVl/kR+ehLo3F5MdP3dMuuRXvwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ57Id2L7L2x766eBCQ7MXObeMKcJs50t/cKoruLpX+zugodSnXSp9Jxxr9xsfMmtYXz/q3xDgwREREpDhMYIiIiUhw+QiIijzNq1CgEBQUBACIiIpCWlubmiIjI0zCBISKPUltbC4DzrBGRbXyEREQe5fTp06iurkZiYiImTpyIY8eOuTskIvJAvANDRB7Fz88PU6ZMQWxsLM6fP4+pU6diz5498PExv1xptVrU1NRAq9XarE9XXNVkmVZ11W4ZS+7cTmp7YuL2VL5GI3Q6ncV1+nrb+xRYclByu7bqDiyxHM+dDBZitxezPboKcW3n6KTfUTTaOOZiaPXi9lHsvjgixD/Eaec6Exgi8ihRUVFo3749VCoVoqKiEBoairKyMrRr186snEajgVarhUajsVlfidC0+6lG09ZuGUvu3E5qe2Li9lS6koNQq9WWV3a2s0/WJoEUw1bdIuvV6XRNY7cXsx3N6d4slsW4HaBRi9tHZ+yLn95P9LleUFDgUN18hEREbx2MsAAAER5JREFUHmXHjh1Yvnw5AKC0tBR6vR6tW7d2c1RE5Gkk3YExGo1ISkpCSUkJDAYDpk2bhkGDBskdGxHdg8aOHYsFCxYgLi4OKpUKy5Yta/L4iIhI0lVh165dCA0Nxdtvv42Kigo8++yzTGCISBa+vr5YuXKlu8MgIg8nKYEZMmQIYmJiTJ+9vb1lC4iIiIjIHkkJTEBAAABAr9dj5syZmDVrlsVyYt88tvVG/mGduN4BYt3ZG6Bx/X3UAaLjcid7cTXnTX8AgC5L0maW3vB3pvLqclHljHU3UV52u2x1vetitMVWzwIxPRZ639fbbhkxvQru7KHgqec9EVFjkh8sX7p0CdOnT0d8fDxGjBhhsYzYN49tvZEvtneAWLZ6Hyilp4DduJrzpn8zNPdNeUedrbguqlx5WTnCW4ffXvCg62K0xRU9C8T0KrizHlvnl6O9BIiInEVSAlNeXo7ExESkpKSgb9++csdEREREZJOkbtSZmZmorKzE2rVrkZCQgISEBNTU1MgdGxEREZFFku7AJCcnIzk5We5YiIiIiEThQHZERESkOBwdioiI5PHjp25pdn/FKVHlOiKo6UJ7MXceKiEiz7Jft9/dITgF78AQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFMcjXuI9rKuSfcTdu1lgyUG3jbbrKmJfyruX3a0v5hERicE7MERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREiuMRvZCIiO4lYnteDu7S1gXR3Dvyq8/hbMV1xzY63IwekQ/2kr4t2cU7MERERKQ4TGCIiIhIcZjAEBERkeJIegemvr4eixcvxo8//ghfX18sWbIE7du3lzs2IroH8fpCRGJIugPzxRdfwGAwYPv27ZgzZw6WL18ud1xEdI/i9YWIxFAJgiA4ulFaWhq6d++OYcOGAQD69euHgwcPmpUpKCiQJ0Ii8ii9ejm3ZwWvL0T3LkeuL5IeIen1egQGBpo+e3t74+bNm/DxuV2dsy9yRHR34vWFiMSQ9AgpMDAQVVVVps/19fVmFxciIql4fSEiMSQlMD179sSBAwcAAMeOHUN0dLSsQRHRvYvXFyISQ9I7MA29BH766ScIgoBly5ahY8eOzoiPiO4xvL4QkRiSEhg5FRYWYsWKFcjJyXFnGGaMRiOSkpJQUlICg8GAadOmYdCgQe4OC3V1dUhOTsa5c+fg7e2NtLQ0REZGujssAMCVK1cwevRobNy40aN+bEaNGoWgoCAAQEREBNLS0twcEbB+/Xrs27cPRqMRcXFxiI2NdXdI2LlzJz766CMAQG1tLbRaLb755hsEBwe7OTL73arz8vKQm5sLHx8fTJs2DU8++aQbozVnL/YlS5bg6NGjCAgIAACsXbvWdL56AmvX53379mHNmjXw8fHBmDFjMG7cODdFaJ212Ddt2oQdO3YgLCwMAPDGG2+gQ4cO7gixCXu/PZ563O3F7bRjLrhRVlaWMHz4cCE2NtadYTSxY8cOYcmSJYIgCMLVq1eFAQMGuDeg//n888+F+fPnC4IgCIcPHxZefvllN0d0i8FgEF555RXh6aefFs6cOePucExqamqEkSNHujsMM4cPHxZeeukloa6uTtDr9cLq1avdHVITixcvFnJzc90dhslnn30mzJs3TxAEQfjhhx/MzvvLly8Lw4cPF2pra4XKykrT357CVuyCIAgTJkwQrly54o7Q7LJ2fTYYDMLgwYOFa9euCbW1tcLo0aOFy5cvuylKy2z9tsyZM0c4ceKEG6Kyz9Zvjycfd3u/mc465m4diTcyMhIZGRnuDMGiIUOG4C9/+Yvps7e3txujuW3w4MFITU0FAFy8eBHh4eFujuiW9PR0TJgwAW3atHF3KGZOnz6N6upqJCYmYuLEiTh27Ji7Q8LXX3+N6OhoTJ8+HS+//DIGDhzo7pDMnDhxAmfOnMH48ePdHYpJQUEB+vXrBwDo0aMHTp48aVp3/PhxPPzww/D19UVQUBAiIyNx+vRpd4XahK3Y6+vrceHCBaSkpGDChAnYsWOHu8K0yNr1+ezZs4iMjERISAh8fX3Rq1cvHDlyxA0RWmfrt6WoqAhZWVmIi4vD+vXrXRyZbbZ+ezz5uNv7zXTWMXfrq/0xMTEoLi52Zwj/v737j4m6/uMA/vyAHCCIFz+KmrId/lgQoKY11AyPH9pMJOdRBhyahtQwycQOYQpWtJAWixhy6NaWWmsK1lrT/DFzBlmHRWzSUiDcpUxgQnBw3MHx+v7hl0+Rx/Ejbp+7fD3+8+5zn/eT9+b787735z7vl1XDy7kGgwE7duzA66+/LnGiv0ybNg0ajQZnz55FSUmJ1HFQVVUFX19frFixAhUVFVLHGcHDwwNbt25FYmIiWlpakJaWhtOnT0v6REtnZydu3bqF8vJy/PHHH3j11Vdx+vRpCIIgWaa/02q1yMjIkDrGCLYeqzYYDCNuuXh5ecFgMEgR0ypb2fv6+pCSkoKXXnoJFosFqampCAsLw6OPPiph4r+MNj47ep8Dtq8tzz77LJKSkuDt7Y3t27fjwoULDnPb0da1x5H7faxrpr36nGshjaK1tRWpqalISEhAfHy81HFGKCwsxDfffIO9e/eir69P0iyVlZWoqamBWq3Gr7/+Co1Gg/b2dkkzDVMoFFi3bh0EQYBCoYBcLpc8m1wux1NPPQWZTIbg4GC4u7vjzp07kmYa1t3djebmZkRGRkodZQRbj1X/873e3l6H+g2Jreyenp5ITU2Fp6cnvL29ERkZ6VCrR6Nx9D63hYiwadMm+Pr6QiaTISoqCg0NDVLHGmG0a4+j9/toue3Z5zyBsaKjowNbtmzB7t27oVKppI4j+uKLL8TlN09PTwiCIPntrWPHjuHo0aM4cuQIQkJCUFhYiICAAEkzDTtx4oS4Df3t27dhMBgkz7Z48WJcunQJRITbt2/DaDRCLpdLmmmYTqfDsmXLpI5xD1uPVUdERODKlSswmUzo6elBU1OTQz12bSt7S0sLkpKSYLFYMDAwgJ9++gmPPfaYVFHHbc6cObhx4wa6urpgNptRW1uLRYsWSR1rXAwGA9auXYve3l4QEX744QeEhYVJHUtk69rjyP1uK7c9+5x3h7KivLwc3d3dKCsrQ1lZGQDg0KFD8PDwkDTXqlWrsGfPHiQnJ2NwcBA5OTlwd3eXNJMjU6lU2LNnD1588UUIgoB3331X8g3RlEoldDodVCoViAj79u2TfBI67Pfff8esWbOkjnGPuLg4VFdXY+PGjeJj1R9//DGCgoIQExMDtVqNpKQkEBF27tzpUP8nxsoeHx+P559/Hm5ubkhISMC8efOkjjyqr776Cn19fXjhhReQnZ2NrVu3goiwYcMGPPTQQ1LHs+nv2Xfu3InU1FTIZDIsXboUUVFRUscTWbv2JCYmwmg0OnS/j5XbXn0u+WPUjDHGGGMTxbeQGGOMMeZ0eALDGGOMMafDExjGGGOMOR2ewDDGGGPM6fAEhjHGGGNOhycw/2EmkwnHjx+3eUx0dDRMJtOYr02WTqcTN+davny5zWPVajVUKhUaGxsn3E5vby/UavWYbTDGptZkxxlbKioqUF9ff0870dHRAIDffvsNOp1uXOfOzs5GfHw8vv/++3G3/3fp6ekIDw+fsjGRTR2ewPyHtbe3jzmw2FtlZSXa2trGfXxhYSHmzp074Xa8vLwcqqI5Y/cLe4wz27ZtQ0RExKjvnzlzZkJfdHbv3o2lS5dOKotWq5V8A0xmHW9k52Sqqqpw/vx5GAwGdHZ2IiMjA6tXr8aPP/6I4uJiuLq6Yvbs2XjrrbdQXl6OxsZGlJaWQqVSIT8/HyaTCV1dXcjIyEBsbKzNtlpbW7F3716YTCa4u7vj7bffhsViwa5duxAYGAi9Xo/w8HDs378fd+7cQVZWFsxmMxQKBS5fvozi4mJcunQJV69exdy5c2E2m7Fr1y7cunULcrkcJSUlcHNzs9r28ePH8dlnn2FoaAgxMTF47bXXEBcXh0WLFuHGjRuIjIxET08P6uvroVAoUFRUZI/uZuy+ZO9x5ty5c6ipqcG+ffug1WpRV1eHgwcP4ssvv0RraytaWlqwZs0aLF68GFlZWeju7kZQUBCAu7tqnzx5Em5ubuLOxfn5+WLto9LSUsycOdPq33XhwgWUlpYCAEJDQ7F//34kJCRgyZIluHbtGhQKBfz8/FBbWwuZTIaKiopRxyjmAKa8vjWzq8rKStq8eTNZLBZqb2+nlStXktlsplWrVlFHRwcRERUXF9Pnn39Oer1eLCdfXV1Nly9fJiKiK1eu0ObNm4mISKlUUn9//4g2hl/LzMykb7/9loiIampq6I033iC9Xk9PPvkk9fT00ODgIK1cuZLa2tqooKCAjh49SkRE3333HSmVSiIi0mg0dPHiRSIiCg0NJb1eT0REKSkp9Msvv4xoNyUlhRobG6mjo4Pi4uLIaDSSxWKhgoICMhgMFBISQjdv3iSz2UwLFy6k69ev09DQECmVSvrzzz+JiGjZsmVT2+GM3YfsPc4YjUZau3YtERG9/PLL9Nxzz9HAwABlZmZSY2OjOG4cOXKEPvjgAyIiqqurE8eVkpIS+vTTT8Vz63Q6Iro73nz99dcj/pbhcw0MDJBSqRTzf/TRR3Tz5k1SKpVUW1tLRESrV68Wx7zk5GRqaGiwmp85Bl6BcUJPPPEEXFxc4O/vDx8fH7S1taGtrU2sANrf33/Pb0ECAgJw8OBBnDhxAoIgYHBwcMx2rl27Bq1Wi8OHD4OIxG8iQUFBYoXdgIAAmEwmNDU1Yf369QCAJUuWWD3fzJkzxa3q/f39YTQarR6n1+sxb948sXRDTk4OgLuFEB955BEAwPTp08VbTTNmzOD704xNMXuOMx4eHlAoFKivr8e0adOwcOFC6HQ6tLa2Ys6cOeJx169fx4oVKwAACxYsGLUUyHBtHX9/f/T391s9prOzEz4+PvDz8wMAbN++XXxveCXHx8dHbN/Hx4fHFQfHExgndPXqVQB3C2gZDAYEBgYiMDAQZWVlmDFjBs6fP4/p06fDxcUFQ0NDAIAPP/wQiYmJiIqKQmVlJU6ePDlmO8HBwdiyZQsef/xxNDU1iT+aEwThnmPnz5+Pn3/+GSEhIairqxNfFwQB9P9qFdY+Z01QUBCam5thNpshk8mwY8cO5ObmjvvzjLF/z97jTGxsLIqKihATE4PZs2ejuLj4nmKiwcHBqKurQ2xsLBoaGsQJkSAIYpvD/x6Ln58furu70dXVBblcjnfeeQfr1q0b9+eZ4+EJjBPq6OjApk2b0NPTg7y8PLi6uiI3Nxfbtm0DEcHLywsHDhyAt7c3BgYGUFRUhGeeeQYFBQXQarV4+OGH0dnZOWY7Go1GvJ/d39+P3NzcUY9NS0vDm2++iVOnTuHBBx8UvyktWLAA77///oSKBPr6+iItLQ0pKSkQBAFKpdJhipYxdr+w9zijVCqRk5ODvLw8BAYGIjMzE/n5+SOOSU5OFguyBgcHi6vAYWFhOHDgwIjVmrG4uLggLy8P6enpcHFxQWhoKMLDwyfVN8wxcDFHJ1NVVYXm5mZkZWVJHWWEixcv4oEHHkBERARqampQXl6OTz75ZELnUKvVyM/Pn9Cg9E/Lly9HdXX1pD/PGHPccWYysrOzsWbNGjz99NOTPkd0dDROnTrlUJXOGa/AsCkya9Ys5OTkwNXVFUNDQzZXa2zRaDR47733JvwodW9vL1555ZVJtckY+28rKiqCm5vbpB6lTk9PR3t7ux1SsX+LV2AYY4wx5nR4IzvGGGOMOR2ewDDGGGPM6fAEhjHGGGNOhycwjDHGGHM6PIFhjDHGmNP5H90XHd6CwDD6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "label_dict = {1: 'Iris-Setosa', 2: 'Iris-Versicolor', 3: 'Iris-Virgnica'}\n",
    "feature_dict = {0: 'sepal length [cm]', 1: 'sepalwidth [cm]', 2: 'petal length [cm]', 3: 'petal width [cm]'}\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(8, 6))\n",
    "    for cnt in range(4):\n",
    "        plt.subplot(2, 2, cnt+1)\n",
    "        for lab in ('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'):\n",
    "            plt.hist(dataset[dataset['class']==lab][dataset.columns[cnt]], #X[y==key, col]       #X[y==lab, cnt],\n",
    "                     label=lab,\n",
    "                     bins=10,\n",
    "                     alpha=0.3,)\n",
    "        plt.xlabel(feature_dict[cnt])\n",
    "    plt.legend(loc='upper right', fancybox=True, fontsize=8)\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Now we standardize the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_std = StandardScaler().fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Eigendecomposition Process\n",
    "    - The eigenvectors represent the directions of the new feature space\n",
    "    - The eigenvalues determine the magnitude of the eigenvectors\n",
    "    - We start with the covariance matrix \n",
    "    \n",
    "$\\sigma_{jk} = \\frac{1}{n-1}\\sum_{i=1}^{N}(x_{ij}-\\bar{x}_{j})(x_{ik}-\\bar{x}_{k})$\n",
    "\n",
    "$Cov = \\frac{1}{n-1}((X-\\bar{x})^T(X-\\bar{x}))$\n",
    "\n",
    "where $\\bar{x}$ is the mean vector $\\bar{x} = \\sum_{x=1}^{n}x_i$.\n",
    "    \n",
    "    \n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Covariance matrix \n",
      "[[ 1.00671141 -0.11010327  0.87760486  0.82344326]\n",
      " [-0.11010327  1.00671141 -0.42333835 -0.358937  ]\n",
      " [ 0.87760486 -0.42333835  1.00671141  0.96921855]\n",
      " [ 0.82344326 -0.358937    0.96921855  1.00671141]]\n",
      "\n",
      "Eigenvectors \n",
      "[[ 0.52237162 -0.37231836 -0.72101681  0.26199559]\n",
      " [-0.26335492 -0.92555649  0.24203288 -0.12413481]\n",
      " [ 0.58125401 -0.02109478  0.14089226 -0.80115427]\n",
      " [ 0.56561105 -0.06541577  0.6338014   0.52354627]]\n",
      "\n",
      "Eigenvalues \n",
      "[2.93035378 0.92740362 0.14834223 0.02074601]\n"
     ]
    }
   ],
   "source": [
    "mean_vec = np.mean(X_std, axis=0)\n",
    "cov_mat = (X_std - mean_vec).T.dot((X_std - mean_vec)) / (X_std.shape[0]-1)\n",
    "#cov_mat = np.cov(X_std.T) # We can also use numpy to compute the covariance matrix as we do in this line\n",
    "print('Covariance matrix \\n%s' %cov_mat)\n",
    "\n",
    "# Now we get the eigenvalues and the eigenvectors:\n",
    "\n",
    "eig_vals, eig_vecs = np.linalg.eig(cov_mat)\n",
    "print('\\nEigenvectors \\n%s' %eig_vecs)\n",
    "print('\\nEigenvalues \\n%s' %eig_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now we sort the eigenvalues (and eigenvectors) from highest to lowest so that \n",
    "we can choose the top k eigen vectors and then reduce the dimensionality.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvalues in descending order:\n",
      "1.  2.9303537755893156\n",
      "2.  0.9274036215173417\n",
      "3.  0.14834222648163986\n",
      "4.  0.02074601399559581\n"
     ]
    }
   ],
   "source": [
    "# We make tuples of (eigenvalue, eigenvector)\n",
    "eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]\n",
    "\n",
    "# Sort the tuples in descending order\n",
    "eig_pairs.sort(key=lambda x: x[0], reverse=True)\n",
    "\n",
    "# Verify our results:\n",
    "print('Eigenvalues in descending order:')\n",
    "count = 1\n",
    "for i in eig_pairs:\n",
    "    print(str(count) + \".  \" + str(i[0]))\n",
    "    count+=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explained Variance\n",
    "In PCA we select the top k components of the new space. All the components in the new space explain 100% of the variance in the data and the k top components explain a smaller percentage of the variance. Here is how we compute the **explained variance** according to how many components we choose. In this example we can see that the first component  explains 73% of the variance, the second 23%, and the third one only about 4%. We could then take the first and second components and drop the rest of them without loosing too much information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.007277045209380133, 0.002303052326768065, 0.000368383195762739, 5.151926808906293e-05]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deVxVdeL/8RdwQVkVxQUrTChSK1Tw2zKm/dzKXLJICVGycmbSplzSRi01MkUtt3TSckJrbBIozalsmSxHxqwmKcIFlxD3DcSFi3JZ7vn90cM7MgJXrQsHeD8fjx5xz+fcc98ckbfn3HM/x80wDAMRERGTca/pACIiIhVRQYmIiCmpoERExJRUUCIiYkoqKBERMSVLTQf4LaSnp9d0BBER+RWioqIuWVYnCgoq/uYuyMrKol27dtWY5tdRXtdSXtdSXterbZmd5a3sIEOn+ERExJRUUCIiYkoqKBERMSUVlIiImJIKSkRETEkFJSIipqSCEhERU3JJQdntdqZNm8bDDz9MfHw8+/fvLzeemppKdHQ0MTExbNiwodzYW2+9xdy5cx2Pv/rqKx566CEefvhhUlNTXRFXRERMyCUf1F2/fj3FxcWkpKSQkZHB7NmzWbp0KQC5ubmsXLmS1atXY7PZiIuLo0uXLtjtdqZMmUJmZib33HMPACUlJcyaNYv3338fb29vhgwZQvfu3WnWrJkrYouIiIm4pKDS09Pp2rUrAB07dmTbtm2OsczMTDp16oSXlxdeXl6EhISwc+dOWrduzQMPPMDvfvc79u7dC0B2djYhISE0atQI+GW2iC1btnDfffe5IraI1DPvfneAVZuP4JN2uqajXJG7rvGgFk0kcdVcUlBWqxU/Pz/HYw8PD0pLS7FYLFitVvz9/R1jvr6+WK1WGjVqxF133cWaNWvKbaeidSuSlZVVaZ6ioqIqx81GeV2rNuX9ZPdZvvr5LO6fHanpKJfNbrfXmrxbjxcBcGuLGg5yhUqKG9aan2G4+r9zLikoPz8/CgsLHY/tdjsWi6XCscLCwnIlVNV2qlq3qnme6tq8VWajvK6TkPYN+06Xcsu1jWs6ymU7d+4cPj4+NR3jstzexofbWrozfuDtNR3litSmn2G4+rn4XFJQkZGRbNiwgb59+5KRkUF4eLhjLCIigoULF2Kz2SguLiY7O7vc+MXCwsLYv38/p0+fxsfHhy1btjBixAhXRBYxrdAmXqQ8cWdNx7hstfGXp5iTSwqqd+/efP3118TGxmIYBomJiaxYsYKQkBB69uxJfHw8cXFxGIbBuHHjaNCgQYXb8fT0ZNKkSYwYMQLDMHjooYdo0aKWHYuLiMhVcUlBubu7M3369HLLwsLCHF/HxMQQExNT4XOjo6PLPe7Rowc9evT47UOKiIip6YO6IiJiSiooERExJRWUiIiYkgpKRERMySUXSUj9URs/iX/u3Llak3fH0bNc30h/TaV+0hGU/Cr/yDjM3vzimo5RZ7UPDuD/hfo5X1GkDtI/zeRX0wdJXUsfJJX6SkdQIiJiSiooERExJRWUiIiYkgpKRERMSQUlIiKmpIISERFTUkGJiIgpqaBERMSUVFAiImJKKigRETElFZSIiJiSCkpERExJBSUiIqakghIREVNSQYmIiCmpoERExJRUUCIiYkoqKBERMSUVlIiImJIKSkRETEkFJSIipqSCEhERU1JBiYiIKamgRETElFRQIiJiSiooERExJRWUiIiYkgpKRERMSQUlIiKmpIISERFTUkGJiIgpuaSg7HY706ZN4+GHHyY+Pp79+/eXG09NTSU6OpqYmBg2bNgAQH5+Po8//jhxcXGMHTuW8+fPA5CUlER0dDQPPfQQX3zxhSviioiICbmkoNavX09xcTEpKSmMHz+e2bNnO8Zyc3NZuXIlycnJJCUlMX/+fIqLi1myZAn9+/fn3XffpX379qSkpHD27FnHusuXLycxMdEVcUVExIRcUlDp6el07doVgI4dO7Jt2zbHWGZmJp06dcLLywt/f39CQkLYuXNnued069aNzZs34+3tTatWrTh//jznz5/Hzc3NFXFFRMSELK7YqNVqxc/Pz/HYw8OD0tJSLBYLVqsVf39/x5ivry9Wq7Xccl9fXwoKCgAIDg6mX79+lJWV8cQTT1T6mllZWZWOFRUVVTluNrUp77lz57Db7bUmL9Su/QvK62q1LS/UvsxXm9clBeXn50dhYaHjsd1ux2KxVDhWWFiIv7+/Y3nDhg0pLCwkICCAtLQ0Tpw4wZdffgnAiBEjiIyMJCIi4pLXbNeuXaV5srKyqhw3m9qU1yftNOfOnas1eaF27V9QXlerbXmh9mV2ljc9Pb3C5S45xRcZGUlaWhoAGRkZhIeHO8YiIiJIT0/HZrNRUFBAdnY24eHhREZGsnHjRgDS0tKIioqiUaNGNGzYEC8vLxo0aIC/vz9nz551RWQRETEZlxxB9e7dm6+//prY2FgMwyAxMZEVK1YQEhJCz549iY+PJy4uDsMwGDduHA0aNGDUqFFMnDiR1NRUAgMDmTdvHj4+PmzevJmYmBjc3d2JjIykS5curogsIiIm45KCcnd3Z/r06eWWhYWFOb6OiYkhJiam3HhQUBBJSUmXbGv06NGMHj3aFTFFRMTE9EFdERExJRWUiIiYkgpKRERMSQUlIiKmpIISERFTUkGJiIgpOS2oY8eOMXr0aPr168ef/vQnDh06VB25RESknnNaUFOmTGHgwIGsWrWKBx98kOeff746comISD3ntKBsNhs9e/YkICCAXr16UVZWVh25RESknnNaUGVlZezatQvA8X8RERFXczrV0ZQpU3juuec4ceIELVq04KWXXqqOXPXW6vRDvJV2BJ+00zUd5bLsOHqW6xu5ZMYsEannnP5mad++PatXr66OLDVmwRe7azqCQ9bRsxy3luBpO1/TUQC4NtC7yvH2wQHc1lIXg4rIb6/Sgho9ejSLFi3irrvuumRs06ZNLg1Vn7ULDqCZp42goGY1HQWAcb3Dna5Tm26cJiK1R6UFtWjRIgDee+89goODHcuzs7Ndn0pEROq9Sgtq9+7dHD9+nLlz5/LnP/8ZwzCw2+3MmzePf/zjH9WZUURE6qFKC+rs2bN88sknnDx5ko8//hgANzc34uLiqi2ciIjUX5UWVOfOnencuTPbt2/n5ptvrs5MIiIizq/iO3bsGPPnz6ekpATDMDh9+jQfffRRdWQTEZF6zOn1wa+99hpPPfUUwcHBPPjgg9x0003VkUtEROo5pwUVGBhIp06dAIiOjubYsWMuDyUiIuK0oDw9Pfn+++8pLS3l3//+N7m5udWRS0RE6jmnBfXiiy9SWlrKqFGjSE1NZfTo0dWRS0RE6jmnF0m8/PLLzJs3D4DFixe7PJCIiAhcxhFUcXExO3fuxGazUVxcTHFxcXXkEhGRes7pEdS+fft48skncXNzwzAM3Nzc+PLLL6sjm4iI1GNOC0qfeRIRkZqg+ySIiIgpqaBERMSULqug9u3bx8aNGzl27BiGYbg6k4iIiPP3oN555x2++OILzpw5wwMPPMCBAweYNm1adWQTEZF6zOkR1Lp163jrrbfw9/fn0Ucf5aeffqqOXCIiUs85LagLp/Tc3NwA8PLycm0iERERLuMUX//+/Rk6dChHjhzhD3/4A7169aqOXCIiUs85Lahhw4Zxxx13sGfPHkJDQ3W7DRERqRZOT/GlpqaSmprKfffdx5w5c1i7dm115BIRkXrOaUGtWrWK8ePHA/DGG2+watUql4cSERFxWlDu7u40aNAA+OXeUBculhAREXElp+9B9ezZk7i4OCIiIti+fTs9evSojlwiIlLPOS2oJ598ku7du5OTk8MDDzxA27ZtnW7UbreTkJDArl278PLyYsaMGbRu3doxnpqaSnJyMhaLhVGjRtG9e3fy8/OZMGECRUVFNG/enFmzZuHt7c3GjRt57bXXAGjfvj0vvPCCjuJEROoBp6f4jh49yqZNm9i7dy/r16/nL3/5i9ONrl+/nuLiYlJSUhg/fjyzZ892jOXm5rJy5UqSk5NJSkpi/vz5FBcXs2TJEvr378+7775L+/btSUlJwWq18sorr/D666+TmprKNddcw6lTp37ddywiIrWC04IaM2YMVquVoKAgx3/OpKen07VrVwA6duzItm3bHGOZmZl06tQJLy8v/P39CQkJYefOneWe061bNzZv3syPP/5IeHg4c+bMIS4ujqCgIJo0aXK136uIiNQiTk/x+fr6Mm7cuCvaqNVqxc/Pz/HYw8OD0tJSLBYLVqsVf3//ctu3Wq3llvv6+lJQUMCpU6f47rvvWLt2LT4+PgwdOpSOHTvSpk2bS14zKyur0jxFRUVVjufl5V/R9+dqpaWl5OXl1nQMALKyypyu42z/mo3yupbyul5ty3y1eZ0W1I033si6deto166d472figriYn5+fhQWFjoe2+12LBZLhWOFhYX4+/s7ljds2JDCwkICAgJo3Lgxt956K82aNQOgc+fOZGVlVfj67dq1qzRPVlZWleNBh3ZX+f1Ut7y8XIKCmtV0DADatQt3uo6z/Ws2yutayut6tS2zs7zp6ekVLndaUFlZWeWaz83Njb/97W9VPicyMpINGzbQt29fMjIyCA//7y+5iIgIFi5ciM1mo7i4mOzsbMLDw4mMjGTjxo1ER0eTlpZGVFQUt9xyC7t37yY/P5+AgAB++uknYmJinEUWEZE6wGlBrVy5stzj4uJipxvt3bs3X3/9NbGxsRiGQWJiIitWrCAkJISePXsSHx9PXFwchmEwbtw4GjRowKhRo5g4cSKpqakEBgYyb948fHx8GD9+PL///e8B6NOnT7myExGRustpQSUnJ7NixQpKS0sxDANPT08+//zzKp/j7u7O9OnTyy0LCwtzfB0TE3PJkVBQUBBJSUmXbKtfv37069fPWUwREaljLmsuvpUrV9KtWzdmzZpVrmhERERcxWlBBQYG0rx5cwoLC7n99ts5c+ZMdeQSESknPj6e7OzsSse///57du7cCcBTTz1VXbEusXjx4irnLF22bBmZmZm/6jUeffTRX/X8i40bN+6y3rqpCU4Lyt/fn/Xr1+Pm5kZycjL5+ea6JFtEBGD16tWcOHEC4LImFKgpf/zjH4mIiKjpGA4LFiww7Y1onb4HNWPGDA4cOMD48eNZvnw5CQkJ1RBLRK7W6vRDpG45+JtuM6bzdTwUdW2l40VFRUyePJkjR45QUFDAzJkzycnJYe/evUyYMAGbzcZ9993HV199RXx8PDfddBN79uzBx8eHzp07s2nTJs6ePcvy5cv58ssvK3zeBceOHSMhIQGbzcbp06f505/+RMuWLfn3v//N9u3bueGGGxg8eDAfffQRQ4cO5ZNPPsHNzY0XX3yR3/3ud4SEhDBjxgwAGjduzPDhw8t9LwUFBTz//POOWWumTJlCQEAAw4cP55133iE7O5vFixfzt7/9jXvvvZcOHTpw4MABbrzxRmbOnOnYTllZGdOmTePYsWOcOnWKbt26MXbsWCZNmkTfvn3Jy8tj48aNFBUVceDAAf7whz8QHR3Nrl27yuVLTEzEx8eHqVOn8vPPP3PddddRUlJSLvPOnTtJTEx0XGH9xBNPMGbMGA4cOMDf//53x3qvvvoqe/bsYe7cuXh6ehITE8OiRYv49NNP2b9/P7Nnz8Zut3P27FmmTJlCZGQk99xzD5GRkeTk5NC0aVMWL15MSUmJ48+7pKSEqVOncsstt/DCCy+wf/9+7HY7Y8eO5fbbb7+aHzeHSgtq69at3HrrrWRkZACQn5/PXXfddcmOERFJTk7mmmuuYcGCBXz22Wf89NNPBAQEVLp+REQEU6ZMYcSIETRs2JAVK1YwceJEvv/+e6evtXfvXh577DFuv/12fvjhBxYvXsyKFSvo2rUrffv2pVWrVgA0adKEm266iS1bttChQwf+85//8PzzzxMXF0diYiI33HAD7733Hh988AGdO3d2bP/111/njjvuIC4ujn379jF58mRWrVrFs88+y6RJk8jLy2PZsmVYLBaOHz/OmDFjaN26NWPGjGH9+vWO7Rw9epSOHTsyePBgbDabo6AuZrVaSUpKYt++fYwcOZLo6GimTp1aLt+bb75Jx44dsdlspKamcuTIET777LNy22nbti02m43Dhw/j6enJqVOnaN++PWlpaSxbtgxvb2+mTZvGpk2baNGiBTabjffeew+ARYsWAfDzzz8zceJEbrrpJj766CPWrFlDZGQkBw8e5O233yY4OJjY2Fi2bt1KRkaG48979+7dbN68maysLAIDA0lMTOTUqVMMGzaMdevWOf3zrEqlBfXNN99w6623VvgCd9111696URFxnYeirq3yaMcV9u7dS7du3QBo3bo1ffr0Yc2aNY5xwzDKrX/zzTcDEBAQwA033OD42mazlVvvf58H0KxZM5YuXcr777+Pm5sbpaWlleaKiYnhgw8+IDc3lx49emCxWMjOzubFF18EoKSkhMDAwHLP2b17N99++y2ffvopAGfPngWgV69eLFiwgN/97ne0bNkSgODgYMdE2J06dSInJ8exncaNG7N161a+/fZb/Pz8Knyf58Lk28HBwY7x/83Xpk0b9uzZ4zgt2KpVqwqnnBs0aBBr167Fy8uL6OhoAJo2bcrEiRPx9fVl7969dOzYEah4soXmzZuzZMkSx2QJF2YDCgwMJDg42JHTZrOV+/MODw8nPDychIQE0tPTHe+vlZaWcurUqUv275WotKD++Mc/Ar/80EyePPmqX0BE6r6wsDC2bt1Kr169OHbsGG+++SY9evQgN/eXKbu2b99+2dtq0KBBlc979dVXGTx4MHfffTerV6/mgw8+AH6ZROB/C+3OO+/klVde4fjx40ybNg345ZfznDlzaNWqFenp6Y6zRBeEhoZy//33M2DAAE6ePOk40li+fDldunRxHEF07NiR48ePk5ubS7Nmzfjhhx8YOHAgO3bsAGDNmjX4+/szffp09u/fT2pq6iX5Krozw//my83NxWKxsG7dOoYPH87x48crvBagb9++PProo7i5ubF8+XIKCgpYtGgR//rXvwB47LHHHK/v7n7p5QczZ85k7ty5hIWFsWjRIg4fPlxpxov/vA8ePMjChQvp0KEDLVu2ZOTIkRQVFbF06VIaNWp0yXOvhNP3oLKzszl79myVh+siUr/Fxsby3HPPMWzYMKxWKy+99BKtW7dm1apVDBkyhJtvvhlfX9/L2lbXrl2rfF6fPn2YOXMmb7zxBsHBwY73ijp06MDcuXO59tr/Hj26ublx7733snnzZseRTkJCAhMnTqSs7Jd5Jh9//PFy2x85ciTPP/88qampWK1WnnrqKbZu3crHH39MSkoKBw8e5OmnnyYlJQUvLy9eeukljh49SocOHejRo4ejoO68806eeeYZ0tPT8fb2pnXr1o6LOKryv/lmzpxJmzZtSE9PZ/DgwbRq1arcfKYX+Pr60rZtW0pLS/Hz88MwDCIjI3nwwQfx8fEhICCAEydOlNs/F7v//vt58sknadq0KS1btqzyzhEX/3mXlZXx3HPPcdNNNzFlyhTHz0BcXFyFRXgl3IyKjqEv0r17d44dO0aTJk0cTbpp06Zf9aK/tfT0dKKioioddzYP1IIvNBdfZcb11lx8NU15XevX5O3SpQtff/31b5zIubq2jyv7He70CGrDhg2/LpmIiMhVcFpQGRkZrFmzxnH13okTJyqckkhEpL6piaOn+sTpCcIZM2Zw2223YbVaadWqFY0bN66OXCIiUs85LaiAgAD69++Pn58fTz/9NMePH6+OXCIiUs85LSg3Nzf27NnD+fPn2bt3r+PyTxEREVdy+h7UpEmT2LNnD/Hx8UyYMIEhQ4ZURy4R+RV+6ytTL+dqTpHfmtMjqM2bN9OlSxeioqJYs2bNbzqLrojUDWlpaaSkpFzWutnZ2cTHxwNVz6Rd0azfNpuNHj16XHE+ZzOMX43qmLW8S5cuv+r5FzPzrOWVcXoEVVpaymOPPUabNm2IiYn51ZP/iUjdc2Hamyu1YMGCSscuzGZTW5ktf1X72qycFtSIESMYMWIEmZmZJCUlMXXqVP75z39WRzYRqSXWrFnD3r17iY2NZeLEibRp04aDBw9y66238uKLL3LixAkmTJiAYRg0a/bfD6H36NGDDz/8kAcffJB//OMf+Pj48Oabb2KxWNi5cyd9+/YlKiqKCRMmcPbsWUJCQhzPjY+PJyEhgbCwMFatWkVeXh5PP/008+bNY9u2bRQWFhIWFsasWbMqzHxh1vLDhw/j4+NjylnL//eI58Ks5Remn6srs5ZXxmlBFRUV8fnnn7N27VoMw2D06NEuCSIidcORI0d499138fb2plevXuTm5rJixQr69+9PTEwMn3zySblTY56entxzzz3885//5IEHHuCTTz4hKSnJcfPBDz74gPDwcMaNG8dPP/3Ed999V+lrW61WAgICWLFiBXa7nX79+lV65fGFWcs7deqEt7e3KWct//zzz8tt58Ks5SdOnKBp06Z1ZtbyyjgtqPvvv597772XhIQEx1xWIiKVadmypWMm7GbNmmGz2dizZw8DBw4EIDIy8pL3bgYPHkxCQgKhoaFcf/315WbA3rNnD127dgV+mW/PYrn019aFGdsaNGhAfn4+zzzzDD4+Ppw7d67SWwRdmLX8/fffx9fX15Szll+YRfxigwYNYsOGDezevbvOzFpeGacF9cknn1T4AyEiUpGKZr8ODQ3lxx9/pG3btmzduvWS8euvvx7DMHjzzTcvuVI4NDSUjIwMevXqxY4dOxy31/Dy8iI3N5ewsDB27NhBixYtSEtL4+jRoyxcuJD8/Hy++OKLCm/ZcWG7999/PzfccAPNmzc35azlFR399e3bl7fffpsdO3bUmVnLK+O0eVROIrWP2S4LHzNmDOPGjeOTTz6pdDbtQYMG8eqrr3LHHXeUWz506FAmT57MkCFDCA0NxdPTE4BHHnmE6dOnExwcTPPmzYFfboS4ZMkSYmJi8PLy4rrrrqt0BvELs5YfO3YMwzBMOWt5RUclvr6+XH/99fj7+9eZWcsr43Q289pAs5m7jmYzr3nK61qXk7emZi2vTF3bx1c8m3lVt17+v//7vyuMJyIicmUqLagLb2IeOHCAkpISbr31Vnbs2IGvry8rV66stoAiIjXNTEdP9UmlBTV//nzglw+bLVmyBIvFQllZmek+fCYiInWT03e2Lp4ctqysjPz8fJcGEhERgcu4im/QoEH069eP8PBwfv75Z55++unqyCUiIvWc04IaOnQoAwcOZO/evVx77bU0adKkOnKJiEg957Sg9uzZwwsvvEBBQQEDBgzgxhtvpHv37tWRTURE6rHLuuX7rFmzaNy4MYMGDWLx4sXVkUtEROq5y/r4b+vWrXFzc6NJkyb4+vq6OpOIiIjzgmrUqBHJycmcP3+edevWERAQUB25RESknnNaUImJiRw6dIjAwEC2bdtW7j4oIiIiruL0Igk/Pz8ee+wxbDYbAOfOnaNx48YuDyYiIvWb04JKSEggLS2N5s2bYxgGbm5uJCcnV0c2ERGpx5wWVGZmJuvXr3fZdOoiIiIVcdo6rVu3dpzeExERqS5Oj6COHj1K9+7dHbc7vpxTfHa7nYSEBHbt2oWXlxczZswod7v41NRUkpOTsVgsjBo1iu7du5Ofn8+ECRMoKiqiefPmzJo1C29vb8f2/vjHP9KzZ89L7rYpIiJ1k9OCmjdv3hVvdP369RQXF5OSkkJGRgazZ89m6dKlwC+Tz65cuZLVq1djs9mIi4ujS5cuLFmyhP79+xMdHc2yZctISUnh0UcfBWDhwoWcOXPminOIiEjtVekpvvfeew+A5ORkUlJSyv3nTHp6Ol27dgWgY8eObNu2zTGWmZlJp06d8PLywt/fn5CQEHbu3FnuOd26dWPz5s0AfPbZZ7i5udGtW7er/y5FRKTWqfQIqmXLlgCEhoZe8UatVit+fn6Oxx4eHpSWlmKxWLBarfj7+zvGfH19sVqt5Zb7+vpSUFDA7t27+fjjj1m0aBGvvfZala+ZlZVV6VhRUVGV43l55rqFSGlpKXl5uc5XrAZZWWVO13G2f81GeV1LeV2vtmW+2ryVFtSFo5kBAwawdetWSktLMQyDEydOON2on58fhYWFjsd2ux2LxVLhWGFhIf7+/o7lDRs2pLCwkICAANauXcvx48cZPnw4hw8fxtPTk2uuuabCo6mq7neflZVV5XjQod1Ov6fqlJeXS1BQs5qOAUC7duFO13G2f81GeV1LeV2vtmV2ljc9Pb3C5U7fg3rqqacoKSnhxIkTlJWV0bx5c/r371/lcyIjI9mwYQN9+/YlIyOD8PD//pKLiIhg4cKF2Gw2iouLyc7OJjw8nMjISDZu3Eh0dDRpaWlERUWVu3vv4sWLCQoK0qk+EZF6wull5larlaSkJCIiIlizZs1lXXLeu3dvvLy8iI2NZdasWUyePJkVK1bw5Zdf0qxZM+Lj44mLi2P48OGMGzeOBg0aMGrUKNatW0dsbCw//vgjw4YN+02+QRERqZ2cHkFdODV3/vx5GjZsSElJidONuru7M3369HLLwsLCHF/HxMQQExNTbjwoKIikpKRKt6k7+YqI1C9Oj6B69+7NX/7yF9q2bUtMTIxutyEiItXism75fsHdd9/N9ddf78o8IiIiQBUF9cwzz+Dm5lbh2NV8eFdERORKVFpQsbGx1ZlDRESknEoL6rbbbgPg5MmTLF26lH379nHjjTcycuTIagsnIiL1l9OLJMaOHUtYWBgTJkzg2muv5c9//nN15BIRkXrO6UUSgGMG8bZt2/LZZ5+5NJCIiAhcxhFUaGgoH374IcePH+err76icePG5OTkkJOTUx35RESknnJ6BLV371727t3rmN0cYNq0abi5ufG3v/3NpeFERKT+clpQc+fOpUWLFo7H27dv5+abb3ZpKBEREaen+EaMGMGmTZsAWL58Oc8//7zLQ4mIiDgtqLfeeovly5fzwAMPcOTIEVJTU6sjl4iI1HNOC2rXrl3k5ubSoUMHsrKyOHbsWHXkEo3Q0UIAABGTSURBVBGRes7pe1CLFy/mjTfeoFWrVmRkZPCnP/2Jjz76qDqyiYhIPea0oP7+97/j4eEBQMeOHVm1apXLQ4mIiFR6im/s2LEAeHh4sHz5csfyJ5980vWpRESk3qu0oE6ePOn4+l//+pfja8MwXBpIREQELuMiCShfSpXdgkNEROS3VGlBXVxEKiUREalulV4k8fPPPzN+/HgMwyj3dXZ2dnXmExGReqrSglq4cKHj64tvXqgbGYqISHVwesNCERGRmnBZF0mIiIhUNxWUiIiYkgpKRERMSQUlIiKmpIISERFTUkGJiIgpqaBERMSUVFAiImJKKigRETElFZSIiJiSCkpEREzJ6S3fRX4LC77YXdMRHPLy8gk6ZI4843qH13QEEdPSEZSIiJiSCkpERExJBSUiIqakghIREVNyyUUSdrudhIQEdu3ahZeXFzNmzKB169aO8dTUVJKTk7FYLIwaNYru3buTn5/PhAkTKCoqonnz5syaNQtvb2/eeust1q1bB8Ddd9/NU0895YrIIiJiMi45glq/fj3FxcWkpKQwfvx4Zs+e7RjLzc1l5cqVJCcnk5SUxPz58ykuLmbJkiX079+fd999l/bt25OSksLBgwf58MMPSU5OJiUlhU2bNrFz505XRBYREZNxyRFUeno6Xbt2BaBjx45s27bNMZaZmUmnTp3w8vLCy8uLkJAQdu7cSXp6Ok888QQA3bp1Y/78+QwdOpQ333wTDw8PAEpLS2nQoEGFr5mVlVVpnqKioirH8/Lyr/h7dKXS0lLy8nJrOgYAWVllTtdxtn/BXPu4Lu5fM1Fe16ttma82r0sKymq14ufn53js4eFBaWkpFosFq9WKv7+/Y8zX1xer1Vpuua+vLwUFBXh6etKkSRMMw+Dll1+mffv2tGnTpsLXbNeuXaV5srKyqhw3y2diLsjLyyUoqFlNxwCgXTvnn9Nxtn/BXPu4Lu5fM1Fe16ttmZ3lTU9Pr3C5S07x+fn5UVhY6Hhst9uxWCwVjhUWFuLv719ueWFhIQEBAQDYbDYmTJhAYWEhL7zwgiviioiICbmkoCIjI0lLSwMgIyOD8PD//isxIiKC9PR0bDYbBQUFZGdnEx4eTmRkJBs3bgQgLS2NqKgoDMPgySef5KabbmL69OmOU30iIlL3ueQUX+/evfn666+JjY3FMAwSExNZsWIFISEh9OzZk/j4eOLi4jAMg3HjxtGgQQNGjRrFxIkTSU1NJTAwkHnz5rF+/Xr+85//UFxczL///W8AnnnmGTp16uSK2CIiYiIuKSh3d3emT59ebllYWJjj65iYGGJiYsqNBwUFkZSUVG5Z79692bp1qysiioiIyemDuiIiYkoqKBERMSUVlIiImJIKSkRETEkFJSIipqSCEhERU1JBiYiIKamgRETElFRQIiJiSiooERExJRWUiIiYkgpKRERMSQUlIiKmpIISERFTUkGJiIgpqaBERMSUVFAiImJKKigRETElFZSIiJiSCkpERExJBSUiIqakghIREVNSQYmIiCmpoERExJRUUCIiYkoqKBERMSUVlIiImJIKSkRETEkFJSIipqSCEhERU1JBiYiIKamgRETElCw1HUBEfr0FX+yu6QgOeXn5BB0yT55xvcNrOoJcJR1BiYiIKamgRETElFRQIiJiSiooERExJRWUiIiYkksKym63M23aNB5++GHi4+PZv39/ufHU1FSio6OJiYlhw4YNAOTn5/P4448TFxfH2LFjOX/+fKXriohI3eeSy8zXr19PcXExKSkpZGRkMHv2bJYuXQpAbm4uK1euZPXq1dhsNuLi4ujSpQtLliyhf//+REdHs2zZMlJSUujXr1+F63p5ebkitohIhcx0GT+Y61J+V17G75KCSk9Pp2vXrgB07NiRbdu2OcYyMzPp1KkTXl5eeHl5ERISws6dO0lPT+eJJ54AoFu3bsyfP5/rrruuwnUjIiIqfE1nmSrTrcnVfJcu1KQhUFDTKQDn+/Vy1zPVPtb+dS0T7V+4vH1cq/YvmGof/1Y/wxVxSUFZrVb8/Pwcjz08PCgtLcVisWC1WvH393eM+fr6YrVayy339fWloKCg0nX/V1RUlCu+DRERqUEueQ/Kz8+PwsJCx2O73Y7FYqlwrLCwEH9//3LLCwsLCQgIqHRdERGp+1xSUJGRkaSlpQGQkZFBePh/z1FGRESQnp6OzWajoKCA7OxswsPDiYyMZOPGjQCkpaURFRVV6boiIlL3uRmGYfzWG7Xb7SQkJLB7924MwyAxMZG0tDRCQkLo2bMnqamppKSkYBgGTzzxBPfeey95eXlMnDiRwsJCAgMDmTdvHj4+PhWuKyIidZ9LCqomFRUV8eyzz3Ly5El8fX2ZM2cOTZqUf4dz5MiRnD59Gk9PTxo0aMCbb75Z7TkvlPiuXbvw8vJixowZtG7d2jGemppKcnIyFouFUaNG0b1792rPeDFneWfMmMEPP/yAr68vAEuWLDHF6diffvqJuXPnsnLlynLLv/rqK1577TUsFgsPPfQQMTExNZSwvMryrlixgvfff9/xs/ziiy8SGhpaExEdSkpKeO655zh8+DDFxcWMGjWKnj17OsbNto+d5TXbPi4rK2PKlCnk5OTg4eHBrFmzCAkJcYybbf86y3tV+9eoY5YvX24sWrTIMAzD+Pjjj42XXnrpknXuu+8+w263V3e0cj7//HNj4sSJhmEYxo8//miMHDnSMXbixAmjf//+hs1mM86ePev4uiZVldcwDCM2NtY4efJkTUSr1LJly4z+/fsbgwcPLre8uLjY6NWrl3H69GnDZrMZ0dHRxokTJ2oo5X9VltcwDGP8+PHG1q1bayBV5d5//31jxowZhmEYRn5+vnH33Xc7xsy4j6vKaxjm28dffPGFMWnSJMMwDOPbb78t93fOjPu3qryGcXX7t87NJHHxJe7dunXjm2++KTeel5fH2bNnGTlyJEOGDKmxD/9e7qX4/v7+jsvra1JVee12O/v372fatGnExsby/vvv11TMckJCQli8ePEly7OzswkJCaFRo0Z4eXkRFRXFli1baiBheZXlBdi+fTvLli1jyJAhvPHGG9WcrGJ9+vRhzJgxjsceHh6Or824j6vKC+bbx7169eKll14C4MiRIwQFBTnGzLh/q8oLV7d/a/X9oN577z3efvvtcsuaNm16yeXqFyspKeHxxx/nkUce4cyZMwwZMoSIiAiaNm1abbnh6i7Fr0lV5T137hzDhg3jscceo6ysjEceeYRbbrmFtm3b1mBiuPfeezl06NAly824f6HyvAD9+vUjLi4OPz8/nnrqKTZs2FDjp30vnM61Wq2MHj2asWPHOsbMuI+rygvm3McWi4WJEyfyxRdfsGjRIsdyM+5fqDwvXN3+rdVHUIMHD+bjjz8u95+/v/8ll6tfLCgoiNjYWCwWC02bNqVdu3bk5ORUe/aruRS/JlWV19vbm0ceeQRvb2/8/Py44447avyIrypm3L9VMQyD4cOH06RJE7y8vLj77rvZsWNHTccC4OjRozzyyCMMHDiQAQMGOJabdR9XltfM+3jOnDl8/vnnTJ06lXPnzgHm3b9Qcd6r3b+1uqAqUtHl6hfbvHmz419OhYWF7Nmzp0beCL2aS/FrUlV59+3bR1xcHGVlZZSUlPDDDz9w880311RUp8LCwti/fz+nT5+muLiYLVu20KlTp5qOVSmr1Ur//v0pLCzEMAy+++47brnllpqORV5eHo8//jjPPvssgwYNKjdmxn1cVV4z7uO1a9c6ToV5e3vj5ubmOC1pxv1bVd6r3b917iq+8+fPM3HiRHJzc/H09GTevHk0a9aMl19+mT59+hAREcHMmTP56aefcHd35/e//z29evWq9pxXcyl+TXKW969//SufffYZnp6eDBw4kCFDhtRo3gsOHTrEM888Q2pqKh999BHnzp3j4YcfdlwBZRgGDz30EEOHDq3pqEDledeuXcvKlSvx8vLizjvvZPTo0TUdlRkzZvDpp5+W+wfe4MGDOX/+vCn3sbO8ZtvH586dY/LkyeTl5VFaWsof/vAHzp8/b9qfYWd5r2b/1rmCEhGRuqHOneITEZG6QQUlIiKmpIISERFTUkGJiIgpqaBERMSUVFBS53333XfceeedxMfHEx8fT0xMzCWTscIvn5tLSUm5om2vWbOGL7/88oqec+jQoRqf2PO39s4779R0BKmDavVURyKX64477mDBggUAFBcX06dPHwYOHFhuppFu3bpd8Xajo6N/s4y12dKlSxk2bFhNx5A6RgUl9Y7VasXd3R0PDw/i4+MJDAzk7Nmz9OvXj/379xMbG8v48eNp2bIlBw8e5NZbb+XFF1/k5MmTTJo0iYKCAgzDYM6cOXz00UcEBQURGhrK66+/jru7O7m5uTz88MMMHTqU//znP/zlL38BfrkVzJw5c/D09Kww15IlS1i/fj1lZWUMGTKE2NhYli9fzrp167BYLHTu3Jlnn32WxYsXs3//fk6dOsWZM2eIi4vjn//8Jzk5OcyZM4egoCDGjBlDs2bNOH78ON26dWPcuHEcOnSI559/ntLSUtzc3JgyZQpt27blnnvuITIykpycHJo2bcrixYux2+288MIL7N+/H7vdztixY7n99tsZMGAAt912G7t27cLNzY0lS5bwzjvvcObMGRISEhg+fDiTJ0/GYrHg4eHByy+/TIsWLarzj1fqEBWU1Avffvst8fHxuLm54enpydSpUx2Thw4YMIDevXuzZs0ax/r79u0jKSkJb29vevXqRW5uLm+88QY9evRgyJAhfPPNN2RmZpZ7jePHj7N27VrsdjsDBgygT58+7Nmzh1deeYUWLVrw+uuv89lnn5WbA+6CHTt2kJaWxnvvvUdxcTHz5s1j165dfPrpp477gj399NOO2fcbNmxIUlISy5YtY+PGjbz++uusXr2adevWMXz4cA4fPkxSUhL+/v7ExcWxfft23njjDeLj4+nVqxdZWVk899xzrFmzhoMHD/L2228THBxMbGwsW7duZceOHQQGBpKYmMipU6cYNmwY69ato7CwkH79+jF16lTGjx9PWloao0aN4p133iEhIYG///3v3HzzzUyaNIktW7Zw5swZFZRcNRWU1AsXn+L7X23atLlkWUhIiGP29mbNmmGz2cjJyXHM4XbnnXcClLs9xoVbpADceOONHDhwgBYtWjBz5kx8fHw4fvw4kZGRFWbIyckhIiICDw8PvL29mTJlCp9++ikdOnRwHHF17tyZPXv2ANC+fXsA/P39ueGGGwBo1KgRNpsNgLZt29K4cWPgl7kdc3JyyM7O5v/+7/8AaNeuHceOHQMgMDCQ4OBgAIKDg7HZbOzevZv09HRHCZeWlnLq1Klyr31h3YsNGjSIv/71r/z+97/H39+fcePGVfj9ilwOXSQh9Z6bm9tlLQsLC2Pr1q0AfP/997zyyivlxrOysigrK+P8+fP8/PPPtG7dmilTppCYmMjs2bNp3rw5lc0sFhoayo4dO7Db7ZSUlPDYY4/Rpk0bMjMzKS0txTAMvv/+e0eZVpTvYtnZ2Zw/f56ysjIyMzO54YYbCAsLc9wzKCsry3G/noq2FRoaSr9+/Vi5ciV//etf6dOnD40aNap0/Qvf15dffklUVBRvv/02ffr0qZG7VUvdoSMokcs0cuRInnvuOT788EMAEhMTWbt2rWP8wgSZp0+fZtSoUTRp0oSBAwcSExNDQEAAQUFBnDhxosJtt2vXjq5duzJkyBDsdjtDhgyhbdu23HfffY5lUVFR9OrV67JuZeLp6cmYMWPIy8ujT58+tG3blj//+c9MnTqV5cuXU1paysyZMyt9fmxsLFOmTGHYsGFYrVbi4uJwd6/837NhYWFMmDCB0aNHO94nc3d3Z/LkyU6zilRGk8WK/Aa+++47kpOTKz2NWJ0unhFdpDbTKT4RETElHUGJiIgp6QhKRERMSQUlIiKmpIISERFTUkGJiIgpqaBERMSU/j/Ryus5buVWVQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "total_variance = sum(eig_vals)\n",
    "var_exp = [(i/total_variance)/100 for i in sorted(eig_vals, reverse=True)]\n",
    "print(var_exp)\n",
    "cum_var_exp = np.cumsum(var_exp)\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "\n",
    "    plt.bar(range(4), var_exp, alpha=0.5, align='center',\n",
    "            label='individual explained variance')\n",
    "    plt.step(range(4), cum_var_exp, where='mid',\n",
    "             label='cumulative explained variance')\n",
    "    plt.ylabel('Explained variance ratio')\n",
    "    plt.xlabel('Principal components')\n",
    "    plt.legend(loc='best')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Projection Matrix\n",
    "Now we need to create the projection matrix that we will use to \n",
    "transform the dataset into the new feature space. This is the\n",
    "matrix of the concatenated top $k$ eigenvectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Matrix W:\n",
      " [[ 0.52237162 -0.37231836]\n",
      " [-0.26335492 -0.92555649]\n",
      " [ 0.58125401 -0.02109478]\n",
      " [ 0.56561105 -0.06541577]]\n"
     ]
    }
   ],
   "source": [
    "# We get the eigenvectors corresponding to the first two (top) eigenvalues\n",
    "# to create matrix W.\n",
    "matrix_w = np.hstack((eig_pairs[0][1].reshape(4,1),\n",
    "                      eig_pairs[1][1].reshape(4,1)))\n",
    "\n",
    "print('Matrix W:\\n', matrix_w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Projection Onto the New Feature Space\n",
    "Now we use our 4x2 projection matrix W to transform our samples onto the new subspace using the equation:\n",
    "$Y = X \\times W$. where $Y$ is a 150 $\\times$ 2 matrix, our transformed samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deVwT19oH8F/YBETF7SLam1CttbTXpWj3W9sKtopy7eKCpqjVt1StisjLtYJapKK26lWvV1S01oVU0NvqrahtBXs/vnZxoYu0pbWiggoiuIARwzrvHzGRwExmkkwmk+T5fj58lEkycwbCPDlnnvMcBcMwDAghhBCZ8XB0AwghhBA2FKAIIYTIEgUoQgghskQBihBCiCxRgCKEECJLFKAIIYTIkpeUB2tsbMSCBQtw/vx5eHp6YtmyZVAqlVI2gRBCiJOQtAf11VdfAQCysrIwe/ZsLFu2TMrDE0IIcSKS9qAiIiLw/PPPAwBKS0vRpUsXKQ9PCCHEiUgaoADAy8sL8+bNw+HDh/HPf/7T5LH8/Hypm0MIIUQGBg4c2GqbwlGljioqKjB27FgcOHAA/v7+APQBqmUjCwsLERoa6ogmSobO0fm5+vkBdI6uQK7nx3btByS+B7Vv3z5s2rQJAODn5weFQgFPT08pm0AIIcRJSDrE9+KLL2L+/PlQq9VoaGhAUlIS2rRpI2UTCCGEOAlJA5S/vz/Wrl0r5SEJIYQ4KZqoSwghRJYoQBFCCJElClCEEEJkiQIUIYQQWaIARQghRJYoQBHCR6MBQkIADw/9vxqNo1tEiFuQvNQRIU5FowFiY4GaGv33xcX67wFArXZcuwhxA9SDIsSc5OR7wcmgpka/nRBiVxSgCDGnpMSy7YQQ0VCAIsQcrgU1aaFNQuyOAhQh5qSlAXer7Rv5++u3E0LsigIUIeao1UBGBqBSAQqF/t+MDEqQIEQClMVHCB+1mgISIQ5APShCCCGyRAGKEEKILFGAIoQQIksUoAghhMgSBShCCCGyRAFKpqg+KSHE3VGauQxRfVJCCKEelCxRfVJCCKEAJUtUn5QQQihAyRLVJyWEEApQskT1SQkhhAKULFF9UkIIoSw+2aL6pIQQd0c9KEIIIbJEAYoQQogsUYAihBAiSxSgCCGEyBIFKEIIIbJEAYq4D6rAS4hTkSzNvL6+HklJSbh8+TLq6uowffp0hIeHS3V44u6oAi8hTkeyHtRnn32GwMBAfPzxx9i8eTPee+89qQ5NCFXgJcQJKRiGYaQ40O3bt8EwDAICAnDjxg2MHj0aeXl5Js/Jz8+Hf4saPzqdDr6+vlI00WHoHO3voUcegYLlrc4oFPjtl19s3r+jz08KdI7OT67nV1NTg4EDB7baLtkQX9u2bQEAWq0Ws2fPxpw5c1ifFxoaavJ9YWFhq22uhs5RAkqlflivBYVSKUq7HH5+EqBzdH5yPb/8/HzW7ZImSZSVlWHixIkYNWoUoqKipDw0cXdUgZcQpyNZgKqsrMSUKVOQmJiI0aNHS3VY4m64MvWoAi8hTkeyIb6NGzeiuroa6enpSE9PBwBs3rxZluOhxEnxZepRBV5CnIpkAWrBggVYsGCBVIcj7shcph4FJkKcDk3UJa6jpMSy7a6AJh8TF0YBirgOpdKy7XLHF3wMQ5rFxQDD3BvSpCBFXAQFKOI6XClTT0jwocnHxMVRgCKuw5Uy9YQEH5kPaWoKNAhZEwKPxR4IWRMCTQH17IhlaMl34lpcJVNPSPDhmHwshyFNTYEGsftjUVOvD7LFVcWI3a/PqFT3dYHfD5EE9aAIkSMh99NkPKSZnJdsDE4GNfU1SM6j4UciHAUoQuRISPCR65CmRoOSmyw9OwAlVfIYfiTOgQIUIXIkNPio1cCFC0BTk/5fGQQnxMZCWcX+sLKD44cfifOgAEWIrTQa9AoPF38uktyCjxB3kzvS8gD/OtOH/L39kRbu+OFH4jwoQBFii7s9Bp+ystbp4FZOonXq7Le7SRzqAiBjP6C6CSgY/b8ZURmUIEEsQgGKuCapKixwpYPHxVk1idaQ/VZcVQwGjDH7zWmCVLMkDnUBcGEN0LQYuLBXRcGJWIwCFHE9UlZY4EoHv3bNqkm0Tp/9JuPMQuJ8KEAR1yNlhQVL5xzxTKLlynJzmuw3uWYWEqdEAYq4HikrLHD1GDp3Zn8+T0DjynJzquw3Z0zuILJEAYq4HimLxt7tMdQFB5v2GNautWqoKy08Df7epq+j7DfirihAEdcj9X0QtRpFeXmmPQYrh7rUfdXIiMqAqoMKCiig6qCSTfZb+5wcWtqDSIpq8RHXYwgCycn6YT2lUh+cpB5qsrIuoLqvWhYByYRGg+BFiwCdTv99y9WKCbED6kER10T3QcSVnAwPQ3AyoKU9iJ1RgCLEkZxlRVyZL+1BXBMFKELsjSsIOdOKuK62WjFxChSgCLEnc0HImVbETUtDk6+v6TaagEvsjAIUIfZkLgg507CZWo2y1FSagEskRQGKEHsyF4ScbNiseuRISjwhkqIARYg9mQtCVLeOELMoQBFiT+aCkIvXrXPqZUOILHAGqB9//BGvvvoqxo8fj1OnThm3v/3225I0jBCXwBeEXHS+ltMvG0JkgTNALV++HKtWrUJqairS0tJw7NgxAEB1dbVkjSPEJZgLQs4yD8pCTr9sCJEFzgDl7e2N+++/H71790ZGRgbef/99/P7771AoFFK2T3Zc9HpCHMGZ5kFZyOmXDSGywBmg2rZtix07dqCurg5du3bFypUrMWfOHFy+fFnK9smKC19PiCM40zwoC7nEsiHE4TgD1MqVK1FVVYW6ujoAQJ8+fbBu3Tr06dNHssbJjQtfT4ilWnSl2+fkWL4PZ5oHZSFaNoSIgTNABQQEYNasWQgICDBue+CBB5Ceni5Jw+TIha8nxBIsXenuf/+7PgnCknFfJ5sHZQk5LxtCnIfkaeY//fQTYmJipD6sKFz4ekIswdKVNt6ZtWTcly0FXaHQ78MFbnCq+6pxYc4FNL3bhAtzLlBwIhaTNEBt3rwZCxYsQG1trZSHFQ3NqyQA+LvMQsd9m6egA/rgxDD6/9MNTkK4A1RjYyPq6uowc+ZM1NfXo66uDrW1tZg4caLVB1MqlVi3bp3Vr7eVrRl4Lj6vkgglpMssdNzXkIKuUt0LTgbOfIOT0l2JCBQM0/KvQm/37t3YuHEjKisr0bVrVzAMAw8PDwwaNAjLly+3+oCXLl3C3LlzsXv37laP5efnw79FF0Wn08G3ZRVlK+TktMeiRcHQ6e7FZF/fJqSmlmHkSMfO7RLrHOXMlc6xfU4Oghctar2AXzN1wcH6ZeAFeuiRR6Bg+VNkFAr89ssvVrVTbEJ/h2w/nyZfX5Slpurr+cmYK71P2cj1/GpqajBw4MDWDzA89uzZw/cUi1y8eJEZM2YM62OnTp1qte3XX38V5bgqFcPoP6KafqlUouzeJmKdo5y53DlmZt57UykUpm8qf3/945aQ8xv0LsG/Q5HPJfN0JqNarWIUKQpGtVrFZJ628GdrAZd7n7Yg1/Nju/YzDMPw3oN65plnsHnzZvzrX/8yfjkjSzPwaISCmGUYmmMYYOdO1AUH2zbu60o3OEVMd6WSSe6NN0DFxcVBq9WiS5cuxi9nZEkGHk3IJRZRq/XDedbU0zN8EoqJAfz8gM6dnf8Gp4jprlQyyb158T2hbdu2iI+PF+2A9913H+v9J3tLS9MHmebZwVwfUM1NyHXG6wWRKcMnIcOb7do1/Zty506nfaNpCjRIflOLknpAWQWk5QHqAljdG6SSSe6NtwfVu3dvHDhwAOfOncP58+dx/vx5KdolOksy8GhCLhEV13ixi5UmMQ7HNVwDowCKA4HYKEDzfGere4NUMsm98fagCgsLUVhYaPxeoVBgx44ddm2UvajVwv5GlEr9sB7bdkIs0rKXZBgv/vpr9jcZ4LSfhFiH43yA5FEBUFvZI0wLT0Ps/liT/VLJJPfBG6B27tyJW7du4fLly/jzn/+Mtm3bStEuh7JkOJAQs7h6SRs3cr/GST8J2WM4zlB9IjkvGSVVJVB2UCItPI2qUrgJ3iG+L774AjExMUhMTMS2bdvcohafNRNyKeuPsOLqDbFPP3TqT0L2Go6TomSSYfXfR3Y/Qqv/yghvgProo4+we/duBAYGYsaMGcjNzZWiXQ5nyUKnQrP+KIi5IUt7Q06auTfjwAxcrL7YarszDMdRKrt88QYoDw8P+Pj4QKFQQKFQwM/PT4p2ORUh97opdd1NcRWEZaNSOW1w2nBqA5qYJpPtbb3bOkUFc0plly/eADVo0CDMnTsX5eXlWLRoEfr27StFu5yKkKy/uDiXStgiQrGNF0+b5ryTclmGATLyM1ifqmvQyT44AZTKLme8AWru3Ll4+eWXMWbMGLzwwgt45513pGiXU+Gbl6jR6Ke4sCkp0dcJpKE/F9ZyvDg93SmrDrfPyWEdBmhkGlmfz7VdbiiVXb54A5RWqzVWkqiqqsK+ffukaJdT4atSY66X1KkTsGhRMA39uRu1Wv8GUSr1n1KSk01/6TK8Ydl19WrWYQDPJvbneyo87d8oEdDqv/LFG6BmzJiBI0eOoKioCEVFRTh37pwU7ZK95teP5GRg0iTuD8R801qaV1gHaOjPLZi7KSmjG5aG7DaPxR7oPa4MGpYR/tiT7K+NHRhr38aJhFb/lS/eeVAMw2DlypVStMVpsM293L6de5SGa+Jv587A9evsx3DSuZpEKL7MGhnU2jJktxkSCEruVoYA7pYvuiv9VxXwdiQy8jPQyDTCU+GJ2IGxSB/hPFNS1H3VUPdVo7CwEKGhoY5uDrmLtwfVp08f/PTTT6irqzN+uTtLK9RwDQGuXUvLyLstc5k1Mqm1xVkZIrzZhrtj2ekj0tGwqAHMuwwaFjU4VXAi8sXbgzpx4gSOHDli/F6hUCDPgoXYXJHQ64dGow9aJSX6e01+fvoek1KpD1qGD8P/8z9NJsN8zpLQRWzAV09LBrW2OLPbOkA/lt3yjSyApkBDVSGIYLwB6rPPPgPDMLh+/ToCAwPh6ekcNz6FaB5ALPlbE1KrT2iharUaKC0tw/r1PSxuB3FifPW0ZFBrS9lBieKq1m90ZaAKaLpg8f5aDhkaJsQCoCBFWPEO8R0/fhwRERGYOnUqhg4diq+//lqKdtmdLfehzWXtRUToP1y+/rrwYcCRI6sFV60gMtEiy659To5lrzdXT8uaWlt2IHZ2G9+E2OYJGVRuiADgX/I9OjqauXLlCsMwDHPlyhVm9OjRYq70a8KeS763ZOuq1IYVvxUK/b+ZmQwTHs6+z+ZfCkXrfcl1GWYxudQ5Zmbql3Vv9ott9PW1fJl3a47b8k1nzXMsOWSz5daDPwi2abl1RYqCQQpafSlSFEzm6UzGP83fZLt/mr/J8aRY+t2l3qcs5Hp+Vi/57unpiaCgIABAUFAQ2rRpY/egKQVb70Oz1eoTcmuOkh9cAEuWjIdOJ3xugDVznIR0+e2Qnt68UGveyDybhuLMTYgV0ruiennuhzdABQQEYOfOnfjtt9+wc+dOdOjQQYp22Z3Q7Dkx50tS8oOLsOXTjbVBREjqqMwXQDQ3ZMhXbojq5bkn3gC1YsUKlJaWYs2aNSgrK8PSpUulaJfd8VV/AMT9QOok1WyIELbMDbA2iAgJijJJT+dibkIsX7khqpfnnngDVLt27RAWFoawsDA89thjLtODEnIfmutaMmnSvSDVvIfl48N+rPBwSn5wKSyfbpp8fYV1j60NIkKCooMn1QlJcuBa24kvIYPq5bkn3gCVnJyMgwcPok2bNti3b5/L9KAA/jWfuK4ZjY36ntSMGaY9rLq61isphIcDbrKElvtg+XRTlpoqfI6CJdsNhHT5hTxHRM0DUpcPuuCNfW9YfY+Ir9wQ1ctzT7zzoM6cOYM9e/YAACZNmoSxY8favVFywTXfCdD3pDIy9MGqOYbRX7cuXLB784gjGdLB76ouLEQPIa+LjAQ2bGDfznc8wPzEPSHPEUnLOU3X7rQu12+4RyQ0scJQbojrMYCWfnc3vAFKqVTi4sWL+POf/4xr164hODhYinbJAttcyuZaBicDmQz5Ezk6eNCy7c21CIpWP0cEbEkLbMS8R2QugBHXxDvE9+OPP2L48OF48cUXMWTIEHzzzTf461//ir/+9a9StM+hDCM5XMUzuLZTKrkb40v7lCqRwZ7LdWg0KLnJMbTQAt0jIrbgDVB5eXn4+eef8eWXX+Knn37CyZMncezYMRw7dkyK9jmcWq2vVM42tB8by76ad3GxbJbwIVISkvYpZHVLWwMLWztiYvRvTlvfmHf3razif6oj7xG1TNiYcWAGValwQrwB6siRI5g5cybefPNN45e74cr4a74wKqB/jGH0/6eFB90QV9pnXNy9780lMlg7r6FlUIuLa90Osd6Yd88xLQ/wb7GwgY+nDzr7dXb4mkpsk3o3nNpAk3ydEO89qPfffx+pqakuk15uLa6hfcP2kJDWCRUOWMKHOBLXMN21a/qA0PxNxJbIEBJi+TpQbIuT8bHljXn3HA3rQSWH66ubK6uAtClbZXGPSMj9MUsTOIhj8Aao3r1744knnpCiLU5N5nMkiRTMpX02Dwhcn3aseROx9dqEsPaN2ewc1QXNFi5UqYDV8rjYC03MoEm+8sc7xBceHo5x48Zh/vz5xi/SGi08SMzONxISEKx5E9kSaKxhyVwreyZqmCE0MYMSOOSPN0Dt3LkTkyZNQmRkpPGLtCbxHEkiR2o10Lkz+2NCAgLXmygykvtCz7Xfzp3v3RxtyZY3ptClQCy8nybmUhtsk3pbokm+ToKvDPqbb74peml1LlIut2EP1q504EznaC1XP0fj+bEsxcH4+wt/M7R8E02fbn5/5o7H9phCod+nLecohAXr2QhZasNSLZfmmJ4zXdBSHW7zPpUZruU2eO9B+fr6YurUqXj44YehuFvHZ+7cuXYPnM5IojmSRM5srebQ8k3ElzhhadIFwwibFIzWy7O//dDbCA0NFXYeFtxPM1ep3NokBprUax8t3xP2rubBG6BeeOEF0Q7W1NSElJQU/P777/Dx8cGSJUug4hqGIMRZiflJRciFXsykC+gvQnGH4kzKFxVXFWPRqUXo3qO7sAsSV8IIy5AkVSp3Di3LWxnS9QHYLUjx3oOKiopCTU0NTp8+jerqaowYMcLqg+Xm5qKurg7Z2dlISEjA8uXLrd6XHDnonjBxZZYmTjR/E3pw/HmbuR+mKdAgdu8U1tp6ukad8PWXLLgpS5XKnYMj1uRSMIxhBh+7pKQktG/fHoMGDcKJEydw8+ZNfPDBB1YdbNmyZejXr58xyD377LP4v//7P+Pj+fn58G/xptbpdPD19bXqeLbIyWmP1au74soVb3TrVo/4+AqMHFlt9vmLFgVDp7t3UfD1bUJqapnZ1wGOO0cpufo52uv82ufkIHjRIv2KvXc1+fqiLDUV1SNH8j6XAdC8wD7Xaw2G7nkal5mbZtsU7B+M+L7xGKli30fz9nRdvRreV66gvls3VMTHsx43pzgHi04tgq7xXrt9PX2ROiiV9xhio/cpt0d2PwIGrcOFAgr8MvYXm9pVU1ODgQMHttrOO8RXXFwMzd2uQEREBKKjo61uhFarRUBAgPF7T09PNDQ0wMvrXjNajnEXFhYKH/cWiUYDpKTcG74vK/NBSkoPdO/eg3PkZvhwoNl1AQCg03lg/foeSEw0X+faEecoNVc/R7udX2go0L27yT0mj7Q09FCrW1dPZ3kTKgB90cimJvOvvau06aZpRGNRVlOGlO9T+If7QkOBxEQAgA+AHne/Wj8tFN17dJdFpXJ6n3JTdlCiuKr1sK2yg9Lmn1l+fj7rdt4hvtraWty5cweAPvo2cpXwFiAgIAC3b982ft/U1GQSnOTCmkVP7TlRl4YO3RzfwmUGXG+2pib+194lpMYeIP7QDtdChkQ+HLEmF2+AmjhxIkaNGoW3334bo0aNwuTJk60+WFhYGI4ePQpAXyX9wQcftHpf9mRNsLHXRF0xl50nLk6EN2Haj51b1dhjGdUBQEkM7oZvUUl74A1Qf/vb37B7925MmzYNWVlZNiVJDB06FD4+PoiOjsayZctkW5XC3N85V28mMrL1arpiTNS1pjdH3JQIs8XV/7MWGV94Q3UTUDCA6ibQ+Q77cymJwf1I3dPlDFBarRYJCQnQarUIDAxEcXExUlNTodVqrT+YhwdSU1ORlZWF7Oxs9OrVy+p92ZO5Cf1svZkZM/RLcjRPN1EogEmTbM82php/RDChVR549qGO/wgX9qrQlKrAhb0qrA2ZTsutE4fgDFDvvvsu+vbti7Zt2wIAhg0bhr/85S9ISUmRqm0Ow/V3fvAge28mI8Om+ZBmUY0/YhGh96ss2Id6ejoyojIQ7B/s8KU0iHvhDFBlZWWYPHmysXqEl5cXpk6diosXL0rWOEdi+zvn6rXYc+l3qvFHJMGTiaPuq0beyDy7D+2IWZNPTlz1vOyNM0B5cEzy8/b2tltj5I6r18I1H7JTp9bbLM3IE2PUhhCzZJKJw7bQYMynMVAsVjj1RZ3tvGjBRGE4A5RKpUJubq7Jtry8PHTt2tXujZIrrt6Mnx/7869dMw1CXNeBnJz2Zo8rxqgNIZxkkonDVqnAMDHU1ou6I3swjqjA4Co4A9S8efOQlZWFV155BbNmzcLo0aORnZ2Nd999V8r2yQpXb8bcenHNP4xyXQdWr3bfoE9kQOpMHI5hBL60dWsv6o7uwVCtQetxzpJt3749tmzZgtLSUly9ehXBwcEICgqSsm2yxFaXMznZ/ErbNTX6jD6ue1VXrrjvsCmRAQsKu9qMbYn6WH3BUa5KBc1Zc1G3R7V0S5irwEDM450H1b17dwwYMICCkxlsQ38tNTa2nidl0K1bvfiNIkQoKTNxzAwnCllo0JqLuqN7MI6owOAqeAMU4dd86M8chmGfzBsfX2G/xhECmM/OkTITx8xwYvNKBYC+CGlz1l7UHV0tXeoKDK6UMUgBSiSGRIbMTPO9KYZpfR1oXu2c6u4R0QnJ0pMqE0ephKYvEDIH8HhX/6+mL4zDiYZKBcy7DHa+ulOUi3pk70jW7Q90esCWM2nFXGCQqgKDo++3iY3zHtS4ceOMc6AMGIaBQqFAVlaW3RvmrAx/11z3nFQq/d9/c4WF+n/ZhuffeAOIiwOuX7d8cVZCAJjP0pP4zaSZF4nYyxtQc/e2a3EgEPs3AD0i0bIlYq2Ke/AP9hnzR84fgaZAI8oxHLGYHxtH328TG2eA+sc//iFlO5ySITOv5Urbhr/55sEG4B/WZ7uO1Nfr09UBk/vJFKSIcDKql5Vce9AYnAxqvPXb7fWW5rrXxIAR7cItl8Dg6PttYuMc4uvRowd69OiBhoYG5OTkYO/evdi7dy82bdokZfssIuXwGNuoSUyMvi4fYNmwfk5Oe4SEmM8ENKBCscRiMqqX5YgLqLl7TWIdVy6BwdH328TGew9q3rx5AIDvv/8ely5dws2b5lfbdBSpJ8Oz9XYYBtiwAejSRR8kk5P1PSZzw/oaDbBoUbCg4GRAhWKJRYRk6Qn9dGfjp0BHXEDTwtNaJVyIfVy5BAZXyxjkDVC+vr546623EBQUhOXLl6OyslKKdllM6snw5oLEtWvsvSo2yckwWSZeCCoUSyzC150X+OmufU6OzZ8CHXEBVfdVY8j9Q1ptt/a4bMkQcgkMjlizyZ54r4wMw6CiogK3b99GTU0NqqoELrkpMamH2YUGCYYBNm7k/hs2177OnQEfH9NtVCiWWMVclp7AT3ddV6+2+VOgIy6gmgINvr30rck2BRSY1H+SxcflypIDIJvA4EqrE/MGqJkzZ+Lw4cMYNWoUwsPDMXjwYCnaZTGph9nT0rgn3rbEMNx/w1ztU6mAykpg61YqFEtEYG5oTuCnO+8rVwQ9j4/UF1CuGn9c2X2W7qt5MoSrBAa54A1Qjz32GIYNG4YuXbrg0KFDxntSciP1shRqNTBtmvAgxfU3nJYG+Po2mWxr3m4qFEtsxjeEJ/DTXX23boKeZ3JcGUzqE5rAoCnQIDwn3OwEV7kkQ7gL3gCl0WgQHR2NjIwMjBs3Dv/5z3+kaJfFHLEsRXo6sHPnvWN27sz9XK6/YbUaSE0to14SsR++ITyBn+4q4uOFfwqUyRIegLAEBsPQXVlNmdkJrnJJhnAXvAFqz5492L9/P9avX499+/Zhx44dUrTLKvbqbfBViTEcs7ISmD6dvZyR4W+YbV8jR1ZTL4nYD98QnsBPd9UjRwr/FGiHrCVrS/hE9o7kLZskdEkMuSRDuAveANW5c2d4enoC0Gf0BQYG2r1RcmLpB8GWvarmf8PWrgdFiE2EDOEJ/XQn9Hk8QdHSYGNtCR9NgQbbf9puXFfKwBB8DK8XOnTnallycqdgGIYx94QpU6bg6tWrePTRR/Hrr7+ioaEBDzygr2G1atUqURuTn5+PgQMHmmwrLCxEaGioqMexBNcEWraSRdbuKzi4DqWlPq0fcCGO/j3am6zPr2UNLUDfrbdwLNmiczTzh6PZn2ZSFgjQ90LMXehD1oSwLlmh6qDChTkXuJvB8bqWx03OS7Zq/85Gru9Ttms/YKbUkcG0adOM/4+KihK3VU5AzPR1rtfQelDErgxBiK0ul72kpXHW+rKmLJC1yQlCF0FMC2cPmjR051icQ3xfffUVAODcuXM4f/68ydfjjz+Oxx9/XLJGOpKY6etcr6H1oIjdqdX6oKFU6oNUcrJ9ExbM3NeyJthYm5wgJHmhpOreUh/B/sE0dCcjnAHKUNKosrISFRUVJl/uRMz0da590XpQxO4ckVXHcb/KmmBjbXKCJYsgqvuqkTcyT9bzmFxprSchOAPUK6+8AkA/rBcSEoKZM2dCp9Ph5ZdflqxxciBm+jrXvpqvB0WIXUhdC8wMa4KNtckJ9loE0RFcba0nIQQVi+3atSsA4LnnnkOyG5bSFjN9nSbeEouJMeFVRktu2BJsrKnUYI9FEB2B697d65++7rK9Kd4kCQB44oknAOirSjQ1NfE8mxAiGrZVLK1ZFEypZM+qs8l38JgAABzMSURBVKUWGNeCaAKItRihEJoCDZLzklFSVQJlByXSwtM4j51TnIPhXwwX9FypmbtH56gFEu2NtwfVvn17ZGdn4/fff8eePXvQtm1bKdpFCAHEG5oTuxaYjCpFmGPJsJimQIPkk8kmz53ynymy6ZnwJXywTSx2drwBavny5Th79ixWrFiBoqIiLF26VIp2EUIA8YbmxK4FJqN7WuYIrRABAHGH4lDfZJpRW9dYh7hDcXZto1BCEj5crSYg7xBfp06dMG3aNNTW1gIAdDqd3RvlrGwY8SCEnZhDc2q1eG9IGd3TMseSlPZrd66xPpdru9QMQ3dck4oB16sJyBugUlJScPToUfzpT38CwzBQKBTIysqSom1ORaxbBYSYMDPh1aHscU/LDpQdlKwXc2e9kBvu3RmGLl19YjHvEN/p06eRm5uLrKwsZGdn2xycDh8+jISEBJv2IUdOMuJBnI0jyvQLIfX6NmaYmxtkLqW95evaerPfX+/sZ2aZAgdxl5qAvD0olUqF2tpa+Pn52XywJUuW4NixY7KsBWUrtg+TgOxGPIgzEnNoTiw2lk+yJLOObz/NexIts9maD4s1PxaAVq/z8fSBAgqTwrLeHt5YO3ytxe2SgpSZkI7CG6DKysrwwgsvQKW6O9HNhiG+sLAwREREIDs726rXy5VGo/9wy1Z2V2YjHoSIx8rAyRdULCGkrh/bhTxkTUir19U11gEAPBWeaGQaoeqgklWauTvirWZ++fLlVtt69Ohhdqd79uzB9u3bTbYtXboU/fr1w/Hjx5GVlYXVq1e3el1+fj78Wwwb6HQ6+Pr6mj2eo4WH90JZGVs1cv2PNji4HvHxFZwVI5zhHG3l6ufo6ucHiHeO4TnhKKspa7U92D8YeSPzWF+TU5yD1QWrcaXmCrr5d0N833iMVI3EI7sfabWUBqCvGPHL2F8428D1OgNfT1+kDkrFSNVIAWfkPOT6Pq2pqbGsmvmePXswZswYZGVlQdFiBb65c+eaPdiYMWMwZswYqxracvhPruXhm7tyhesR/c+trMwHKSk90L17D9YPnM5wjrZy9XN09fMDxDvHK7vZ/2Cu1Fxh3b+mQIOU71OMPZ6ymjKkfJ+C7j26m02CMNdWrtcZ6Bp1WP/beiQOS+Q7Haci1/dpfn4+63bOJIlu3boB0N+Duv/++02+iCkhw3iUMEGInqXFYs0N49mziKyrzSlyRpwB6tlnnwUAHDx4EK+88orJl7tpXgqtSxf9V/OyaGwJTWwoYYIQy4vFmpvLJFYRWTbOmoruSnjTzNu1a4e8vDwUFRUZ14OyxRNPPMF6/0muWlZ0uXZN/9W8ugtgmgns6cm+r06dpGs3IXJlaVDh63HZWkQ289VMq3phxP54s/iuX7+Obdu2Gb9XKBTYsWOHPdskK2zzm5ozDN01r0yu0QBvvAHUt1iH8NYt/WNyyxgmRGqWpEjbe7VbrlR0yt5zPLMBSqvVIiMjQ5Q5UM5KyLBcy+eo1UBcnL6n1VxdnT6YUYAiRDgpAoghYMo1icBdcQaozMxMbN26FV5eXli4cKHxnpS74aro0vI5LV2/zv5cug9FiOXcYVIqaY3zHlROTg4+//xzZGVltZrT5E74EiC4qrtwZfbRxF1CCBGGM0D5+PjAx8cHnTp1Qn3LmylupGUptM6d9V98ZdFkVKqMEEKckqAVdXmKTbg8ayq62FiqjBBC3B5ngDp79iwSEhLAMIzx/warVq2SpHHOTo41PgkhxFlwBqg1a9YY/x8dHS1JYwghhBADzgD1+OOPS9kOQgghxARvJQkiveallQzllAghxN0ISpIg0qGl4wkhRI96UDJDS8cTQogeBSgR8Q3NCRm646o0QRUoCCHuhob4RMI3NGfu8bCwe/vhKq1EFSgIIe6GelAi4RuaEzp0RxUoCCFEjwKUSPiG5oQO3bUsrWSunBIhhLgyGuITCd/QnCVDd1SBghBCqAclGr6hORq6I4QQy1CAEgnf0BwN3RFCiGVoiE9EfENzNHRHCCHCUQ+KEEKILFGAIoQQIksUoAghhMgSBShCCCGyRAGKEEKILLldgKK1lgghxDm4VZo5rbVECCHOw616UFKstUQ9NEIIEYdbBSh7r7Vk6KEVFwMMc6+HRkGKENelKdAgZE0IPBZ7IGRNCDQF9AcvFrcKUFxrKom11hKthuveLl26hLFjx5psO3r0KLKzs0U/1uHDh1FeXi76follNAUaxO6PRXFVMRgwKK4qRuz+WApSInGrAGXvgq20Gq7zkGoodvDgwRg3bpzo+92xYwe0Wq3o+yWWSc5LRk296afSmvoaJOfRp1IxuFWShCERIjlZHzSUSn1wEitBglbDdQ72TpaJiYlBx44dUV1djREjRqC4uBizZs1CXFwctFotdDodEhMT8cQTT5i87ssvv8TmzZvh5eWFHj164IMPPsDt27eRnJyMGzduAAAWLFiAsrIyFBYWYt68efj444+RmZmJAwcOwMvLC4MGDUJiYiLy8/Px/vvvw8vLC+3bt8fKlSsBAMnJybh16xZu3LiBMWPGYMKECbafsBsrqWL/9Mm1nVhGsgB169YtJCYmQqvVor6+Hu+88w4effRRqQ5vZM+CrWlpphc+gJbUkCNzQ7FivTeioqIwdOhQfPrppwCAkpISVFZWYtu2bbh27RouXLjQ6jU5OTmYPHkyRowYgX379kGr1WLTpk148sknMWHCBFy4cAHz58/Hrl27EBoaipSUFJw/fx6HDh1CVlYWvLy8MGvWLHz11Vc4ceIEhg4diqlTp+LIkSOorq7GjRs3MGLECLz44osoLy9HTEwMBSgbKTsoUVzV+lOpsgN9KhWDZAHqo48+wpNPPonJkyfj3LlzSEhIwN69e6U6vCTs3UMj4pBiKPb+++83+b53795Qq9WYO3cuGhoaEBMTg1OnTmHt2rUAgKlTp2L+/PnYtGkTdu3ahZ49eyIiIgJnzpzBd999h0OHDgEAqqurTfZ77tw59O/fH97e3gCAQYMG4Y8//sC0adOwceNGTJo0CUFBQejXrx+6dOmC7du348svv0RAQAAaGhrEO2E3lRaehtj9sSbDfP7e/kgLp0+lYpAsQE2ePBk+Pj4AgMbGRrRp04b1eYWFhSbf63S6VtvkLCwMuHstMeJrvrOdozXkdI7duvVCWZkPy/Y6FBYWWbVPnU6H8vJy3LlzBw0NDTh//jzq6+tRWlqKa9eu4dChQygqKkJ8fDyuX7+Od955BxkZGUhKSjLuY/369RgxYgTGjRuH9PR0bN++HYGBgRg4cCCee+453Lx5E4cPH0ZhYSHu3LmDoiJ9W48fP46ff/4ZHh4eOHLkCF544QVs3rwZ/fv3x9/+9jf8+9//xvr161FTU4Pg4GAMHz4cBQUFyM3Nteh3Iqffob1Yeo5hXmFICUvB6oLVuFJzBd38uyG+bzzCvMJk+bNytt+hXQLUnj17sH37dpNtS5cuRb9+/VBRUYHExESTP8zmQkNDTb4vLCxstc3V0DlKa8UK9qHYFSt8rG5jYWEhunbtCj8/P7Rp0wY9e/ZEr169UFhYCJ1OhyFDhuDQoUNYuHAhvL29kZCQ0OpYQ4YMwbJlyxAYGIi2bdsah9+Sk5Px9ddfQ6vVYubMmQgNDcUzzzyDjRs3YuvWrbhy5QoWL16MpqYmDBw4EJMnT8bp06exePFi+Pv7w9vbG6mpqbh8+TJSUlJw8uRJBAYGwtfXF7169TJ+cBRyjnL5HdqLNecYGhqKxGGJdmqRuOT6O8zPz2d/gJHQb7/9xkRGRjL//e9/WR8/depUq22//vqrvZvlcHSO0svMZBiVimEUCv2/mZm27U9u52cPdI7OT67nx3btZxiGkWyI7+zZs4iLi8OaNWvw0EMPSXVYQljR6saEyJ9kAWrVqlWoq6tD2t2UtoCAAGzYsEGqwxNCCHEykgUoZwtGGg1l4xFCiCO51URdoajqOSGEOJ5blToSimrqEUKI41GAYkE19QghxPEoQLGwd9Vz4pqkrGbOJyMjA6dPn7boNTExMcbJv4TIAQUoFvauek5kQKJy5vaqZs4nNjYW/fr1k/y4hIiJkiRYUE09F2fnLBhrqpnX19cjMjIS//nPf+Dv748tW7bAy8sLL730EhYuXIja2lq0adMG7733HhobGzF9+nQEBgZi8ODB8Pf3x759++Dh4YGwsDDMmzcP77zzDiIjI/H4449j/vz5KC0tRX19PRYuXIi//OUvSEpKwsWLF9HY2Ig33ngDkZGRxrZUV1cbCzs3NjYiLi4OTz31FGbPno0+ffrAx8cH//jHP2z+ORHChwIUB5rI6cIkKGduaTVzb29vvPjii/jyyy/x8ssv4+DBg/jwww+xePFixMTE4LnnnsO3336LlStXIj4+HhUVFfjkk0/g4+OD1157DQsXLsSAAQPw8ccfmxSBzcrKQo8ePbB69WqcOXMG33zzDX755Rd07NgRK1asgFarxauvvoonn3zS+JoNGzbg6aefxqRJk1BeXo7x48cjNzcXd+7cwYwZM/Dwww+L8jMihA8FKOJ+JMiCsaaa+ZgxY5CSkoKePXsiJCQEHTt2xJkzZ7Bp0yZs2bIFDMMYq5bfd999xhp6y5Ytw9atW7Fy5UoMGDAADMMYj3vu3DkMHjwYAPDggw/iwQcfxOLFi/H0008D0E+Y79WrFy5evGh8TVFREaKiogAAQUFBCAgIwPXr11nPixB7ogBF3I8EK0sqFAqT73///Xfcvn0bGRkZuHr1KqKjo3HkyBHs3LnT5HkMw2DLli0YP348AKBnz56YMmUKwsLCUFRUhJMnTwIAPDzu3T7evXs3Fi9ejDZt2mDq1Kn44YcfjI/16tULBQUFiIiIwMWLF7FmzRo8+uijOHXqFIYOHQqtVoszZ87gvvvuM3nNqVOn8PDDD6O8vBzV1dUIDAxsdVxC7I0CFHE/DlhZMiQkBOvXr8e+ffvg7e2N2bNnsz5v9OjRWLt2rXHIbd68eUhJSUFtbS10Oh2SWSbj9enTB6NHj0bHjh0RFBSE/v37G4cWo6OjkZSUhNdffx2NjY1ISkpCnz59sHDhQowfPx61tbWYOXMmOnfubNzfW2+9haSkJHzxxRfQ6XRITU2FlxddKoj0FEzz8QAHy8/Px8CBA022ybU8vJjoHB1A5FpWsjs/O6BzdH5yPT+2az9APSjirigLhhDZowFlQgghskQBihBC7ExToEHImhB4LPZAyJoQaArsMzHc1dAQHyGE2JGmQIPY/bGoqdcn5RRXFSN2v35iuLovDTObQz0oQgixo+S8ZGNwMqipr0FyHi2PwIcCFCGEiIRtKK+kin0CONd2cg8FKEJEYo9q5pZUJec71qeffoq8vDyr20LMMwzlFVcVgwFjHMrr5NeJ9fnKDrQ8Ah+6B0XckqZAg+S8ZJRUlUDZQYm08DS73A8wlBmyVqyhiK0Ix3r11Vdtagsxj2soz8/LD/7e/iaP+Xv7Iy2clkfgQwGKuB1737QWs5r5b7/9hsjISFRWVuKTTz5BU1MTZs+ejUuXLkGj0aBDhw7w9vY2ViM/d+4coqOjkZCQgG7duuHixYvo27cvFi9ejHXr1qFLly4YN24clixZgtOnT6O+vh6zZs3CCy+8gEWLFuHKlSu4ceMGBg8ejDlz5tj8s3AnXEN21+9cx85Xd0rygcjV0BCfCCRaWoiIRIqb1lFRUdi2bRs8PT0B3KtmvnHjRqxatQo6nc7k+c2rmQPAwYMHMWrUKJPntG/fHrt27UKfPn2wZcsW7Nq1C1u3bsWdO3daHf/ChQtIS0vDnj17cPToUVRUVBgfy8vLw40bN/Dvf/8bW7ZsQUFBAcrKyjBgwAB8+OGH2LVrF3bt2iXaz8JdcA3ZKTsooe6rxoU5F9D0bhMuzLlAwUkg6kHZyM5LCxE7kOKmtVjVzNn2WVJSgl69esHPzw8A8Oijj7Y6vlKpREBAAACga9euqK2tNT52/vx5DBgwwPhYfHw8tFotCgoK8N133yEgIAB1dXUi/STcR1p4mknPHKChPFtRgLKRBEsLEZEpOyhRXNW6mrmYN63FqmbenKGSuFKpxLlz56DT6eDj44PTp0+jZ8+eZo/fXM+ePfH5558DAG7duoU5c+bgueeeQ7t27ZCamori4mLs3r0bDMOY3Q8xZegV0VCeeChA2UiCpYWIyBzxSdfaauZsOnXqhDfffBMTJkxAYGAgamtr4eXlZbJQoTnh4eH49ttvMX78eDQ2NuLtt99G9+7dMXfuXOTn58PPzw8qlQpXr15FUFCQVefrrtR91RSQxMTIyKlTp1pt+/XXXx3QEuFUKoYBWn+pVML3IfdzFIPczjHzdCajWq1iFCkKRrVaxWSezrRpf1KeX319PZOenm78fsKECcyJEyfsfly5/Q7twdXPUa7nx3btZxiGoR6UjRywtBARgTN/0vXy8sKdO3fwyiuvwNvbG/369cOgQYMc3SxCREcBykaG+0wiLi1ECK+5c+di7ty5jm4GIXZFAUoEtLQQIYSIj+ZBEUIIkSUKUIQQQmSJAhQhhBBZkuweVE1NDRISElBVVQU/Pz+sWLECnTqxV/klhBBCJOtB7d69G4888gg+/vhjjBgxAunp6VIdmhBCiBOSrAc1efJkNDY2AgBKS0vRpUsXqQ5NCCHECSkYhmHE3umePXuwfft2k21Lly5Fv379MHHiRJw5cwYfffQRQkNDTZ6Tn58vdlMIIYQ4gYEDB7baZpcAxaeoqAhvvfUWcnNzpT40IYQQJyHZPahNmzZh3759AAB/f3/jOjmEEEIIG8l6UJWVlZg3bx7q6urQ2NiIhIQE1i4dIYQQAjhoiI8QQgjhI/uJujU1NZg+fTomTJiAqVOn4vr1645ukuhu3bqFadOm4fXXX8e4cePwww8/OLpJdnH48GEkJCQ4uhmiampqwqJFizBu3DjExMSguLj1Qoiu4KeffkJMTIyjm2EX9fX1SExMxIQJEzB69Gjk5eU5ukmia2xsxPz58xEdHQ21Wo0SJ1mwTvYByh3mT3300Ud48sknkZmZiWXLliE1NdXRTRLdkiVLsGrVKjQ1NTm6KaLKzc1FXV0dsrOzkZCQgOXLlzu6SaLbvHkzFixYYLJsvCv57LPPEBgYiI8//hibN2/Ge++95+gmie6rr74CAGRlZWH27NlYtmyZg1skjOyrmbvD/KnJkyfDx8cHgP6TTps2bRzcIvGFhYUhIiIC2dnZjm6KqPLz8/Hss88CAAYMGICff/7ZwS0Sn1KpxLp16/D3v//d0U2xi2HDhuGll14yfu+KCVwRERF4/vnnATjXdVRWAUro/ClnZu4cKyoqkJiYiKSkJAe1znZc5xcZGYnjx487qFX2o9VqERAQYPze09MTDQ0N8PKS1Z+WTV566SVcunTJ0c2wm7Zt2wLQ/y5nz56NOXPmOLhF9uHl5YV58+bh8OHD+Oc//+no5ggj0Yq+ojh79iwTHh7u6GbYxW+//cZERkYy//3vfx3dFLv57rvvmDlz5ji6GaJaunQpc+DAAeP3zz77rANbYz8XL15kxowZ4+hm2E1paSnzyiuvMHv27HF0U+zu6tWrzPPPP8/cvn3b0U3hJft7UO4wf+rs2bOIi4vDqlWr8Nxzzzm6OcQCYWFhOHr0KADgxx9/xIMPPujgFhFLVVZWYsqUKUhMTMTo0aMd3Ry72LdvHzZt2gQA8PPzg0KhcIprqezHIV577TXMmzcPn3zyCRobG7F06VJHN0l0q1atQl1dHdLS0gAAAQEB2LBhg4NbRYQYOnQovv76a0RHR4NhGJd8f7q6jRs3orq6Gunp6cYkrM2bN8PX19fBLRPPiy++iPnz50OtVqOhoQFJSUlOca+b5kERQgiRJdkP8RFCCHFPFKAIIYTIEgUoQgghskQBihBCiCxRgCKEECJLFKCIUzt+/DieeuopxMTEICYmBmPHjsXOnTtbPe/o0aMWl1n69NNPLS4ceunSJYwdO7bV9qqqKiQlJUGtViM6Ohrx8fG4deuWRft2tOzsbNTX17M+5oqFgInjyX4eFCF8nnzySaxevRoAUFdXh2HDhmHUqFFo37698TmDBw+2eL+vvvqqaG2cO3cuoqOjMXToUADAtm3bsGjRImO7ncGmTZvw8ssvt9q+ZMkSHDt2DKGhoQ5oFXFlFKCIS9FqtfDw8ICnpydiYmLQsWNHVFdXY8SIESguLkZ0dDQSEhLQrVs3XLx4EX379sXixYtx7do1vPPOO7h16xYYhsH777+P/fv3o0uXLujZsyc2btwIDw8PVFRUYNy4cVCr1Thx4gT+9a9/AQB0Oh3ef/99eHt7t2rT5cuXUVlZaQxOABATE4PXXnsNgL6a9vbt2+Hj44OQkBCkpqZi//79+Oqrr6DT6VBRUYGJEyciLy8Pf/zxB/7+978jIiIC4eHh6N+/P0pKStC7d2+kpaVBq9UiMTERWq0WjY2NiIuLw1NPPYWoqCg8/vjj+P3336FQKJCeno527dph1apVOHnyJBiGweTJkzF8+HDExMTgoYcewh9//AGtVou1a9fim2++QUVFBeLj41utKOCqhYCJ41GAIk7vu+++Q0xMDBQKBby9vbFw4UJjAdCoqCgMHToUn376qfH5Fy5cwIcffgg/Pz9ERESgoqICmzZtwpAhQzB+/Hh8++23OH36tMkxysvLsW/fPjQ1NSEqKgrDhg3DH3/8gRUrViAoKAgbN27E559/jqioqFbtu3r1Ku677z6TbZ6enmjXrh1u3LiBdevWYe/evQgICMDSpUuRnZ0Nf39/3L59G1u3bsWBAwewbds27N69G8ePH8eOHTsQERGB8vJyxMXFQaVSIS4uDrm5ufjhhx/w9NNPY9KkSSgvL8f48eORm5uL27dvY8SIEVi4cCESEhJw9OhRBAQE4NKlS8jKykJtbS3Gjh2LZ555BgDQr18/JCcnY/Xq1Thw4ABiY2OxYcMG1h6fqxYCJo5HAYo4veZDfC3df//9rbYplUpjBfKuXbuitrYW58+fN9Zhe+qppwAA69atM77m0UcfNS6J0rt3b5SUlCAoKAhpaWnw9/dHeXk5wsLCWNvQvXt3XLlyxWRbfX09Pv/8c6hUKjzwwAPG9jz22GM4duwY+vfvbxwya9euHXr16gWFQoEOHToY12UKDg6GSqUytu/8+fMoKioyBsmgoCAEBAQYF/l8+OGHja+rra1FaWkpfvnlF+NChA0NDSgtLTV5brdu3VBZWcl6XoTYGyVJEJemUCgEbevVqxcKCgoAACdPnsSKFStMHi8sLERjYyPu3LmDs2fPQqVSYcGCBVi6dCmWL1+OP/3pT+CqGhYUFISOHTsiNzfXuG3Hjh3Izc3Ffffdh6KiItTU1AAATpw4YQyqbO1srry8HBUVFQCA77//Hg888AB69eqFU6dOGR+vrq5GYGAg6/569uyJJ554Ajt37sT27dsxfPjwVj295hQKhcstOEnkjXpQhACYNm0akpKS8NlnnwHQr2FlqKIP6HsXb775Jm7evInp06ejU6dOGDVqFMaOHYv27dujS5cuuHr1Kuf+P/jgA6SmpmLr1q2or6+HUqnEkiVL0K5dO8yaNQsTJ06Eh4cHlEol/vd//xcHDhzgbbOPjw/ee+89lJWVoX///hgyZAgGDhyIpKQkfPHFF9DpdEhNTeVcm2rIkCE4ceIEJkyYgJqaGkRERJisbdXSoEGDEBsbix07dvAGT0LEQMViCeFx/PhxZGVlyS7j7plnnsHXX3/t6GYQYjc0xEcIIUSWqAdFCCFElqgHRQghRJYoQBFCCJElClCEEEJkiQIUIYQQWaIARQghRJb+HwD9a3Q2LW3NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Y = X_std.dot(matrix_w)\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(Y[y==lab, 0],\n",
    "                    Y[y==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Same PCA process but using the Scikit-learn library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1xUdf4/8NdwExAVb4uXdsa0cq2vWmpl9S1LsLyRWabohLr5jdTygnxZE9RVCrXUr7puXtDMC5OoXdy8dBFqf25tabKVbkuZKOAFEdRAxBGE8/tjnJGBc+Z65syZ4fV8PHgoh5lzPgcG3vP5fN6f90cjCIIAIiIilQnwdgOIiIjEMEAREZEqMUAREZEqMUAREZEqMUAREZEqMUAREZEqBSl5sdraWsydOxenTp1CYGAgFi9eDK1Wq2QTiIjIRyjag/ryyy8BAFlZWZg+fToWL16s5OWJiMiHKNqDiomJweOPPw4AOHfuHNq1a6fk5YmIyIcoGqAAICgoCLNnz8aBAwfwl7/8xeprubm5SjeHiIhUoG/fvo2OabxV6qi0tBSjR4/Gvn37EB4eDsAUoBo2Mi8vDz169PBGExXDe/R9/n5/AO/RH6j1/sT+9gMKz0Ht3r0b69evBwCEhYVBo9EgMDBQySYQEZGPUHSI78knn8ScOXOg1+tx48YNpKSkoFmzZko2gYiIfISiASo8PByrVq1S8pJEROSjuFCXiIhUiQGKiIhUiQGKiIhUiQFKBgYD0KULEBBg+tdg8HaLiIh8n+ILdf2NwQAkJABVVabPCwtNnwOAXu+9dhER+Tr2oNyUmnorOJlVVZmOExGR6xig3FRU5NxxIiJyDAOUm6R2C+EuIkRNj+GYAV1WdkHAwgB0WdkFhmOckHYHA5Sb0tOBm6UELcLDTceJqOkwHDMgYU8CCssLIUBAYXkhEvYkMEi5gQHKTXo9kJEB6HSARmP6NyODCRJETU1qTiqqaqwnpKtqqpCak8qelYuYxScDvZ4BiaipKyoXn3g296TMwcv8OQDoe/IPhy3sQRERyUDbSnziOVATKNmzItsYoIiIZJAenY7wYOsJ6fDgcNQKtaKPl+px0S0MUEREMtD31CMjNgO6VjpooIGulc7yuRipHhfdwjkoIiKZ6HvqReeV6s9BAaaeVXo0U33tYQ+KiMiDpHpWTJCwjz0oIiIPk+pZkW3sQRERkSoxQBERkSoxQBERkSoxQBERkSoxQBERkSoxQBERkSoxQBERkSoxQEkwGIAuXYCAANO/BlbHJyJSFBfqijAYgIQEoOpmZZLCQtPnALfVICJSCntQIlJTbwUns6oq03EiIlIGA5SIIokq+FLHiYhIfgxQIrQSVfCljhMRkfwYoESkpwPh1vuOITzcdJyIiJTBACVCrwcyMgCdDtBoTP9mZDBBgohISczik6DXMyAREXmTYgGqpqYGKSkpOHv2LKqrqzFlyhRER0crdXkiIvIxigWojz/+GJGRkVi6dCkuX76MkSNHMkAREZEkjSAIghIXunr1KgRBQEREBC5fvoxRo0YhJyfH6jG5ubkIb5CdYDQaERoaqkQTvYb36Pv8/f4A3qM/UOv9VVVVoW/fvo2OK9aDat68OQCgsrIS06dPx8yZM0Uf16NHD6vP8/LyGh3zN7xH3+fv9wfwHqUYjhmQmpOKovIiaFtpkR6drtrt3dX6M8zNzRU9rmgWX3FxMcaPH48RI0YgNjZWyUsrhjX8iJoOwzEDEvYkoLC8EAIEFJYXImFPAgzH+IsvB8UCVFlZGV588UUkJydj1KhRSl1WUeYafoWFgCDcquHHIEXkn1JzUlFVY10XraqmCqk5rIsmB8UC1Lp161BRUYE1a9YgPj4e8fHxMBqNSl3ewpM9HNbwI2paisrF659JHSfnKDYHNXfuXMydO1epy4nydJVy1vAjalq0rbQoLC8UPU7ua1KVJDzdw2ENP6KmJT06HeHB1pnH4cHhSI9mXTQ5NKkA5ekeDmv4ETUt+p56ZMRmQNdKBw000LXSISM2Q7VZfL6mSZU60mpNw3pix+VgHiZMTTUFPa3WFJxYMonIf+l76hmQPKRJ9aCU6OHo9UBBAVBXZ/qXwYmIyDVNKkCxSjkRke9oUkN8AKuUExH5iibVgyIiIt/BAEVERKrEAEVERKrEAEVERKrEAKUgVjonInIcA5SMbAUgVjonInIOA5RM7AUgVjonInIOA5RM7AUgVjonInIOA5RM7AUgZyqdc66KiIgBSjb2ApCjdQA5V0VEZMIAJRN7AcjROoCcqyIiMmlytfg8xZGtNhypA8i5KiIiEwYoGclRiNbTe1YREfkKDvGpDHflJSIyYYBSGe5ZRURkwiE+FeKeVURE7EEREZFKMUAREZEqMUB5ECtCEBG5TjJA/fDDD3j22WcxduxYHDlyxHL8lVdeUaRhalI/0LRrZ/qwF3RYEYKIyD2SAWrJkiVYvnw50tLSkJ6ejq+++goAUFFRoVjj1KBhoLl40fRhL+iwIgQRkXsks/iCg4Nx++23AwAyMjLw4osvon379tBoNIo1Tg3EAk195qDTMOuOFSGIiNwj2YNq3rw5tm7diurqarRv3x7Lli3DzJkzcfbsWSXb53WOBBSxx7RpI/5YVoQgcp7hmAFdVnZBwMIAdFnZBYZjHCtvCiQD1LJly1BeXo7q6moAQPfu3bF69Wp0795dscapgSMBpeFjDAZAbCQ0JIQVIYicZThmQMKeBBSWF0KAgMLyQiTsSZA1SJkD4D0772EAVBHJABUREYFp06YhIiLCcuyOO+7AmjVrFGmYWoiVHqrPXIaofiLFhAlATU3jx7ZowQW4RIBzPaLUnFRU1ViPs1fVVCE1J9Xpc0m1xdMBkFyjeJr5jz/+iPj4eKUv67KGpYfatjV91C9DBFgnUtTWip/r0iXl2k2kVs4GhKJy8XH2ovIit4KLObC98OELNgMgeY+ipY42bNiAjz/+GGFhYUpe1m32Sg916WI7kcKM809EtntE+p6Nf9G0rbQoLG9c4l/bSuv0uczMga3hc+uTCoykHMkeVG1tLaqrq/Hqq6+ipqYG1dXVuH79OsaPH+/yxbRaLVavXu3y89XKkUQKViQnMrHVIxKTHp2O8GDrcfbw4HCkR6c7fS4zscDWkLYV31F6m2QP6oMPPsC6detQVlaGwYMHQxAEBAQEoF+/fi5f7KmnnsKZM2dsPiYvL8/qc6PR2OiY2nTo0A3FxSEiXxEAAB071iAxsRR9+lRA7FZ84R7d5e/36O/3B8h3jx3CO6C4qlj0uNj5+wT1wYI+C7Di2AqcrzqPDuEdkNgzEX2C+jh9LjN7ASw0MBSv/OEVv/uZ+trrVDJAjR49GqNHj8b777+PUaNGKdagHj16WH2el5fX6JjaLF0KxMeb5p+saaDTAQUFIQA63/xozBfu0V3+fo/+fn+AfPe49MbSRsNr4cHhWDp4qeT5e/TogeTBybKcC5AeNgSAQE0gjLVGvP3z2+jUuZPNoUJfo9bXaW5uruhxu0kSjzzyCDZs2IC//vWvlg+ypteLBScTLswlv+VisUl9Tz0yYjOga6WDBhroWumQEZvhUiBw5FxiWX5iw4YhgSHQQINawZTlVFheiD/u/iOz+bzIboCaMWMGKisr0a5dO8sHNabTiR9nYgS5TY1Vh90sNqnvqUfBzALU/bkOBTML3Oql2DqXVJYfgEaBLTggGAKs32nW1NVgxiczXG6bJzWFxct2s/iaN2+OxMRE2S542223YefOnbKdTy3S002/n/Wz+ZgYQW4zBwLzC8scCADvLqqzVWxSRYv9bGX5NQxmmoXiZdwuXrvo0Ta6omEWYv3A609DknZ7UHfeeSf27duHkydP4tSpUzh16pQS7fI53KqdPEKtVYd9pNikq1l+atXU1m7Z7UHl5eVZZX1oNBps3brVo43yVdyqnWQnZyAwGEyBrajINPacnu76C1arNfXmxI6riK01VA21DWsr2ltqG9bWI21zVlNcu2U3QG3btg1XrlzB2bNn8fvf/x7NmzdXol1EBMgXCOQeKvSRMe306HTRLL/06MbtXDVkFf64+4+oqbtVpywkMASrhqxSpK32NMW1W3aH+D777DPEx8cjOTkZmzdvbnK1+Ii8SqwYpCuBQO6hQh8Z03YmY1DfU4/0+9OtHrtpxCbVzOnY6x1JBV5fZjdAvfvuu9i5cyciIyMxdepUZGdnK9Euv6XGhCxSMbkCgSfmjPR6oKAAqKsz/etEm5TMQKuf5ZcenY7UnFTJ6w7XDZctu1ButnpH7qTqq5ndABUQEICQkBBoNBpoNBqfq6MnB7mCilRm7t69LeVsLvkbNwKBhdSQoBfmjFwt8OpqUDM/T7NQg/gP4322arlUyafMZzNVF0zlYjdA9evXD7NmzUJJSQnmz5+Pnj17KtEu1XBzuYcVqVGWFSvay9NYIilyDRXKwN72GWLcCWrm5wFotM7JlzLf5Fzg7CvsBqhZs2bhmWeewfPPP48nnngCr732mhLtUg05h+6lRlPOnw92/mREzvDGnJHE0IMrqd+uBDWp50ld13DMgOi90ape+CrnAmdfYDdAVVZWWipJlJeXY/fu3Uq0SzXkHLqXGk3p0EFkd0MiOcmZYu7o9SSGHqTmUmzNsbi6nsmRtGttK62lp1VcVeyTw3/+ym6Amjp1Kr744gvk5+cjPz8fJ0+eVKJdqiHn0L3UKEtiYqnzJyNylJzj1I6yMfRga/sMKa4ENUe+br6uqz008iy7AUoQBCxbtgxJSUlISkrCrFmzlGiXargydC+VVCE1yjJ8eIWnmk/knWoUNoYeXJlLcSWoST3PrP51nemhNYUaeGphN0B1794dP/74I6qrqy0fTYm9ofuGwSgmxrT1htSbVTkSsoic4sg4taOpqo4+zs7Qg7NzKa4mCOh76jGh9wRoYF1nzxzczM93tIfmzhbz5Dy7Aerw4cNITEzE4MGDMXjwYAwZMkSJdrnEU2uMpIKK2MhJTk7jrTfsvVndu7cl10aR59gbp3ZwCLDl3r2ODxV6IGvQ1QSB/b/ut5u952gPjUOByrJb6ujjjz+GIAi4dOkSIiMjERgYqES7nOaNos8zZjQeOZEi9SbWYADmz+8Io9H0uVqKVZMfsVeWyMHK5O1XrHC8grn5cyUTMyQ4MnxnDnbJnybjfNV5aFtprXpYzpyL5GO3B3Xo0CHExMRg0qRJGDRoEL7++msl2uU0pYfZDQbgohNV+KXexKamAkaj9Y+hqgp44QVTb2rqVFaeIDfZG6d2MFU1+Px5hx5ndV0VjGc7Onyn76lHzvAcmz00V5M1yDV2A9TKlSvx3nvvYffu3di+fTtWrlypRLucpnT1f2cCn0YjPbJhq32FhcDatcomX5GfshUsHExVrenQwaHH2aN0koHY8J0GGgy9c6gs5zIPBTJ5Qn52A1RgYCCioqIAAFFRUWjWrJnHG+UKpSu5OBr4NBpg8mTpN4/Otk8NWwGRD7I1QevgfFFpYqLb80reSDLQ99TjodsesjomQMCWH7c4fV2pZA0Aqkme8KdAaTdARUREYNu2bfj555+xbds2tGrVSol2OU3pSi5SgSUiwnokZds2wFYB+PR0IDS0zqlrq2xPOFI7e0kQDlaZqBg+3O1qFN5IMjAcM+CLU180Ou7qdcWSNdSSPOFvWYZ2A9TSpUtx7tw5rFy5EsXFxVi0aJES7XKa0pVcpALiunW3RlLS0029HVvzR3o9kJZWDJ3O8WurbE84UjtHJmgdnS9yc17JG0kGqTmpjbL45L6uWpIn1BIo5WI3QLVo0QJ9+vRBnz59cP/996u2BwUoOyfryPooRzNyhw+vQEEBkJnZOOg1pMI94UjtVLQ9uzaojVPH5WArSMiV3KCW5Am1BEq52A1Qqamp2L9/P5o1a4bdu3ertgflDbbWR02Y4HxWoVjQmzJF9XvCkdqpaKuN9GwgvMFa//Bq03FPkQoSGmhk2+DP1UoXclNLoJSL3QB1/PhxrFixAhMmTMCqVavwww8/KNEun2XuOdXWin+9sNB22njDoLdmjSoydcmXqWirDf3/u4SMPYDuN0AjmP7N2GM6DlhP8Ld7qx3avdXO7cl+qWy9gbcPlK0auFq2wlBLoJSL3QCl1Wpx+vRpAMDFixfRsWNHjzfKF5mTpF54wfbiXY2GGxaSwhyZoFVqq2etFvpjQMFKoG6h6V/9MdPxhhP8F69dxMVrF92e7N//637R4ycunXDzZqzZqnShVGadWgKlXOxWkvjhhx8wZMgQdOrUCSUlJQgJCcF///d/AwC++uorjzfQFzSsYiFFoxEvg7RiRXskJ3uufUTQ66W730qWYbFR1cLe3k3myX5n/9h6e17GHHjN92YOtgA8Ejj0PfU+G5AashugcnJylGiHTxNLkmooMFB62I8bFpJXOVjqSBY2SiAVLYy3+3RXgoq2ldayo27D40qwlVnnL4HEU+wO8X3xxRd49dVX8dJLL1k+mjKxkRB7yVDh4cCWLZBMJeeGheRVSmf5SWQXORIwXAkq3p6X8XYPzpfZ7UG9+eabSEtLU3V6uVKkRkLCw4GrV8Wfo9NZ18gUG90wbVjY2aNtJ5Kk1ZpezGLHFZQenW41FNaQq0HF3EtJzUlFUXmRZCFYT/F2D86X2e1B3XnnnXjwwQfxhz/8wfLRVEmNhFy7Jv74tm2tM+9c3bBQqflraqJUkuXXcIK/bVhbtA1rK8tkv6tbdcjB2z04X2a3BxUdHY0xY8aga9eulmOLFy/2aKPUwGBoPEwuNeJRJ1Gp6NKlxsfE5qrz8my3Q+ltRKiJUdHWGP40wW/m7R6cL7MboLZt24b/+Z//QYsWLZRojypIBYU2bcS32JBKgJBjhETJ+Wtqwmxl+cGUiZb8aTLO75TeK4mk+WPgVYLdIb527dph6NChePTRRy0f/k4qKADiIyHmeaj6NBpgqPPV/BtRUZUa8gVyjAc3OIdh7VQk7ElAcVWxXxQgJd9htwcVGhqKSZMm4e6774ZGowEAzJo1y+MN8yapP/6XLpmqk0uNhKxbd2udkyCYMvceecS9no5K5q/JF8gxHmwwwLDij0gdWYOiVoC2vBCVBWtR1eANGNOkSQl2e1BPPPEEhg8fjq5du+L222/H7bff7vLF6urqMH/+fIwZMwbx8fEoFPvLqwK2SpdJ1d/bv198Ea67ezepZP6afIEM20obNs5AwlM1KIwEBA1QGAlcDBN/LNOkmx6l95qyG6BiY2NRVVWFo0ePoqKiAsOGDXP5YtnZ2aiursaOHTuQlJSEJUuWuHwuT3IlKHhqKE7pbUTIh8nwIky99yKqQhoc1Ig/Vs40aX/aZM9feWOvKY0gNHzfby0lJQUtW7ZEv379cPjwYfz222946623XLrY4sWL0atXL0uQe/TRR/GPf/zD8vXc3FyEN4gMRqMRoaGhLl3PHXv3tsSKFe1x/nwwOnSoQWJiqc108OjobigubvibDXTsWI2cnHyb1/LWPSrJ3+/Rk/fXcu9etF+xAsHnz6OmQweUJiaaNg9soFt0NEKKixsdFwICAEGw+Vyze3bcDUEiINUXGhiKtH5pGK6TPpej9hbuxfwj82GsNXrk/M7g61Ra9N5oFFc1fn11DO+InOHuVRyqqqpC3759Gx23OwdVWFgIw82J1piYGMTFxbnciMrKSkRERFg+DwwMxI0bNxAUdKsZPXr0sHpOXl5eo2NK6NED9erjhcC0kFZ6Me3SpeKLcJcuDbHbfm/do5L8/R49dn8GA7BggeWFFVJcjM4LFqBzp06Nu9FiL0IAmpvrIGw+9yZtcFsU3hBJVa1H10rnWBaf2FoNkesO+WyIVXACAGOtEW///DaSBytbpJKvU2nnd54XP1513u3vWW5uruhxu0N8169fx7WbK1GNRiNqpQrKOSAiIgJX65VcqKurswpOvoxDceQRzswrNXwRBgY2foydOan0p1chXNN4JMBM10rn2EJXJ3bsZCkg3+CNvabsBqjx48djxIgReOWVVzBixAhMnDjR5Yv16dMHBw8eBGCqkn7XXXe5fC41UnJHX2oinJ1Xqv8ilFpBbmNOSt9Tj4yRm9A2rG2jr4UGhjpe/cCJwOpvm+z5K29UxLAboJ5++mns3LkTkydPRlZWlltJEoMGDUJISAji4uKwePFizJkzx+VzEamWnLWpHNkNV+p6Lu6kq++pR9mfypD5bKbVvkJp/dIcTyt3IrCyFJBv8MpeU4KEK1euCLNmzRKuXLkiCIIg7NmzR5g5c6blc084cuRIo2P/+c9/PHY9uWVmCoJOJwgajenfzEzHnudL9+gqf79Hy/1lZgpCeLggmAa2TB/h4Y6/GBq+iKZMsX0+W9cT+5pGYzqnO/foCJ3O+rrmD51O/LaPZgq6FTpBs0Aj6FbohMyjDn6/JDQ835S9Uxw6f5N5naqM2N9+QRAEyQA1a9Ys4d133xXq6uoEQRCEmpoaYePGjUJSUpJnWij4doBy5++Sr9yjO/z9Hi3317atU3+YrUi9iKZMkX7nIxUI2raV/pozAVPsHh15J+ZuoHZD5tFMITw9XMACSH6Ep4eLBqkm8zpVGakAJTnEV1xcjIkTJ1qqRwQFBWHSpEmW7d/JmgxrJMnXGQzixRoBx9YiSb2I9u+XntyUOu/Fi+IlSMzndPWF6WjygxezhuztzAvcqoRB6iYZoAICxL8UHMzdX8WwZh7Z/KPvSG0qV15Erta8cvWF6WxWoReyhhzN/mOWoPpJBiidTofs7GyrYzk5OWjfvr3HG+WLXJyPJn9i649+/TIkciY1iJU9cYQMgc3QE+gyEwj4M9BlZKFqqj84mv3HLEH1kwxQs2fPRlZWFkaOHIlp06Zh1KhR2LFjB/785z8r2T7VkPqbYj5eWGgayaiPNfOaGKk/+m3b3uo92Boic6XGlthQWtvGKeJOndOWm/do6AkkxMKqZt+Lf3sR7d5q5/VyRWJZgQ0xS9A3SK6SbdmyJTZu3Ihz587hwoUL6NixI6KiopRsm2pIFYn++mtTxXLzcUEw/Y0QhMZbvVMTkJ4uXk5k1apbn9saIisouPUYZzYObLiXU8MXLCDfC/PmPaZGVzWq2VddW42L10xzcOY6bQAUr3gutkHg0DuHYv+v+7lhoI+xW8ahU6dO6NSpkxJtUSWDAZgwofGGhFVVpjeuDY+b/waY/9ZQE+LIzrT25pnsbBwoWzvcPHfRry/Yfag3t+TgBoH+we5C3abM/EZUqrqT1HEmRjRh9hID2rQRf57UcU+1w81zayN1Dj2UiQjkDgYoG8RGY+oTK3UGMDGCPMSRChVyVrGwwZF5HoBbcpB7JAPUmDFjEBcXZ/VhPuYv7P0u2+oJ2drq3ZwwodEAMTFyt5pUocGLp+XevY4979Il547Xv5699UdOFGh1V8OyN23D2iI4wHoJirOJCLYCkDf2IiIVkFrZe+bMGckPJVcTe2rlsyML3aUW4gcGWleaMS+qDwkRf3x0tO22qHV1t5z86h5FXjy1oaGOVUlwsgSQU89z9dwOsvczdKdckVj1h/rVHnQrdKIVIXQrdO7cUiN+9ToVodb7c7qSROfOndG5c2fcuHEDe/fuxUcffYSPPvoI69evVzJ+eowj6w2lsn63bLk1pF9/qL+6WvxaOTkeHW0hpYm8eAKMRseqM7iSSg44tojXy6vF9T31KJhZgLo/10luySHVSxKr/lC/2gO35Gia7M5BzZ49GwDwr3/9C2fOnMFvv/3m8UYpwZHfZTmrtXhwtIWU5k4gcPVF5cgiXpWvFrc1TGcvAHFLjqbJboAKDQ3Fyy+/jKioKCxZsgRlZWVKtMvjHP1dljMZirX5/IS7gcCVF5UjPS9Xe2cKsdVLsheAuCVH02Q3QAmCgNLSUly9ehVVVVUoLy9Xol0e5+7vsliCRXS0/ecxBd0PiLx46kJDPRsIHOl5eaBAa/0huei90W4lJdjqJdkLQF7Zi4i8z97k1eHDhwWDwSBkZ2cL/fv3F5YsWSL3/JiF0tttuLp/k60Ei+ho8XlqW/PVb711xqV2+BK1Ts66rMGL58xbb7l9DqsfvKsvThnZS1xwlr1EB7n3hHKF371OG1Dr/Tm9H1R9Fy9eFH744Qfh8uXLsjaqIV/ZD8qRZClHt8PJzBSE0NBab2yboyg1/hzl5PT9ObvRoBdeFHJnzskd8DyBr1PvcDqLz8xgMCAuLg4ZGRkYM2YM/va3vynRsVM1VxIs2rYFwsKA+HjrjL7UVMBotP4xcK6qCbCVRqqSzcUkh+R+K3RpITCH6chZdmvx7dq1C3v27EGzZs1w7do1vPDCCxgxYoQSbVMtrVZ8LzixBAu9XrrYLOD1zGDyFld+8Aq/KLSttCgsb/xC15bDeiEw4PA8F2vkkTPs9qDatm2LwJs1fUJDQxEZGenxRqmdWIJFcDBQWSn+xlLqDfGMGarPDCZPsfWDV8mLQjRxoRpIz6l34GbPbuq+qQhKC4JmoQZBaUGYum+qom0l/+RQFt8zzzyDefPmYdSoUSgrK0NSUhKSkpKUaJ8qiQ3faTSmXbbFKszY2pV76FAgNLTO6riKMoPJU2ylkaokXbzhkJz2NyBjD6A/Zv24qXcXYu2RtagVTNWTa4VarD2y1qeClDlb8Z6d97DOn4poBEEQbD3g8OHDkl974IEHZG1Mbm4u+vbta3UsLy8PPXr0kPU6cjNvWNiQedsNqa+bH/PKK2fx9tudZd8ZQU184efoDpfuz2CQ3hLD1te8pLpTJ4QUFzc6HjQfqBV5qxuoCcSN+TcUaJl7zAuI66/RCg8O98v5MbX+Hor97Qds9KC+/PJLAMDJkydx6tQpq48HHnhA9uDky+xNJ9h641tUBAwfXuGxnRFIpewFIE9ul+Gi0sRE0Z6dWHACYOlRqZ29MkvkPZIBylzSqKysDKWlpVYfZM3elIFeL70LN+eamoCGq7qnTlWs6ricKoYPF10IHKgR33dG6rjasM6fekkGqJEjRwIAYmNj0aVLF7z66qswGo145plnFGucr3BkymDVKlVMK5DSxLbAWLdOFWnkLhHp2SX0TRB9aGhQqE/M5bDOn3o5VCy2ffv2AIABA9/brWAAABzNSURBVAYg1Rd+iRTmpSo05AvEUjilpn0LC1XfixKzZtgaTOk3BQEa6z8nV2uu+sSeTazzp14O7aj74IMPAgDuv/9+1NXV2Xm0f3B2Y1JHpgxUOK1Anubs2iUfGOoTs2bYGvy+5e8bHfeFuRwuIFYvuwGqZcuW2LFjB3755Rfs2rULzZs3V6JdXqXgxqTk76QmGTUa8eO+MtQnwlNzOUps9W7ey+qn0T9J7mVFyrMboJYsWYITJ05g6dKlyM/Px6JFi5Rol1eppNIM+QOpCcrJk6Wf46NlRDwxl8Ot3ps2uwGqTZs2mDx5MtLS0jB+/HgYjUYl2uURjg7bsfwQyUZq8nHNGtP/xfhoaqdk5Ym/Vbo8/MAU8KbNbi2+BQsW4ODBg/jd734HQRCg0WiQlZWlRNtkZaseXsO5IEdr7RE5xFyUsaH0dOsXJeDTqZ3mYbHUj2egqOYitOWmskj6YxeBw87V7DNjCnjTZrcHdfToUWRnZyMrKws7duxwOzgdOHDAK2WSnBm2U0mlGfJ3jpa89yH6nnoUbIhA3UKgYGW9skgujpEzBbxpsxugdDodrl+/LsvF3njjDSxfvtwrmYDODNsxJZycYjCgW3S0S1tQWFI7t20Drl2TLujoS2QcI2cKeNNmd4ivuLgYTzzxBHQ3x8vdGeLr06cPYmJisGPHDpee7w5nh+2kRmWIANwqVVRYCGg0CDGvbXJhCwoAtrv4vvZClHGM3DJsmJOKovIiaFtpkR6dziy7JsJusdizZ882Ota5c2ebJ921axe2bNlidWzRokXo1asXDh06hKysLKxYsaLR83JzcxHeYGzNaDQiNDTU5vUcsXdvS8yf39Fqc8DQ0DqkpRVj+PAKt8/vDrnuUc386R5b7t2LjvPnI8BGwlB1x47Iz8mR/HpDf7jnHmhEfhUFjQY///STS+2Um6M/Q7HvT11oKIrT0kzlklTMn16nYtR6f1VVVaLFYiW3fN+5c6cgCIKwbNkyYfny5VYf7vj222+FmTNnOrztr5xbFGdmmrZl12hM/7qyg7Yc52hIrdswy8mv7lGns96OXexDo5HnnDqdB27ANU79DD3xi6IAv3qdilDr/Ult+S45xNehQwcApjko84aFvs7dYTtnMgHJjzkyl+LocFaDoUKrMki+nJnDMXKSgWSAevTRRwEA+/fvx6ZNmxRrkJr50zQBuUFqjsXM0cDS8B2PINwKUjqdKvaAIvImu1l8LVq0QE5ODvLz8y37QbnjwQcfFJ1/8gVcwEsARNchWPo9zqR8ShWSNe906ePBSYkSReTf7GbxXbp0CZs3b7Z8rtFosHXrVk+2SbW4gJcA3Aoc9TYcPPfKK+icnOzcefz4HU/DXWrNJYoAMAOPHGYzQFVWViIjIwNhYWFKtUfV/GzhP7mjwRxLRV4ebOe2ivDjdzy2ShQxQJGjJIf4MjMz8fTTT2PEiBH4xz/+oWSbVIsLeElWflyyhCWKSA6SAWrv3r349NNPkZWV1WhNU1PGPZ1INn78joclikgOkgEqJCQEISEhaNOmDWpqapRsE5F/sVVG30/f8bBEEcnBoR11BdvFJohIir3dL53dutlHcJdakoNkksSJEyeQlJQEQRAs/zdbvny5Io0j8nn2yuj78cpvfU89AxK5RTJArVy50vL/uLg4RRpD5HdspZL72Mrvlnv3AkOGWFLruZCYPE0yQD3wwANKtoPIP9lKJfeldVAGAzrOnw+YC8D6WW+P1MmhOSgicpGtVHKp9U5qXAeVmtq4eruLmxASOYoBisiTbKWS+9I6KF/q7ZHfYIAi8jSpVHJfWgflS7098hsMUETe5CvroNLTUddwozu19vbIbzBAkX/y0/VFXqPXozgtzTd6e+Q3GKDI/9hbHKtkO1wIkmrdpqJi+HDf6O2R32CAIv9jb3Gs3AwGdIuOtg5ELgZJ8zYVheWFECBYtqlQS5AiUhIDFPkfJTPObgaikOJi60A0Y4ZLQdLWNhVETQ0DFPkfJTPOpHprFy+KP95OkPSLbSo4/0cyYYAi/6Pk+iJne2V2gqTPb1Ohlvk/8gsMUOR/lFxfJBVw2rZ1KUj6/DYVSs//kV9jgCL/pNT6Iqne2qpVLgVJn9+mol6P0tAT6DITCPgz0GVkIRM9yGmSxWKJyAE3A051cjJCzp9vXOXbhcDo09tU3CyOa+gJJMQCVSGmw4WRQMIeU3FZn703Uhx7UETu0uuRn5Mjf2/NkWQDtSUk3OxRpkbfCk5mzEYkZ7EHRaRG5mQDW5sZOvIYpd28btGvL4h+2aeyEcnr2IMiUiNHkg3UmpCg10MbqRP9ks9kI5IqMECRf1HbkJerHFlsrOItMHw+G5FUgQGK/Ic/rcFxZLGxirfA8PlsRFIFBijyH2od8nKFI4uNVb7hob6nHgUzC1D35zoUzCxgcCKnMUCR/1DxkJfTHFls7EsbHhK5gAGK/IeXh7zOnDmD0aNHWx07ePAgduzY4doJbSw2PnDgAEpKSnxnw0MiFzBAkf9wYshLqVyKxx57DGPGjJH9vFu3bkVlZaXs5yVSE66DIv9h7j2kppqG9RpWdbjJ08uH4uPj0bp1a1RUVGDYsGEoLCzEtGnTMGPGDFRWVsJoNCI5ORkPPvig1fM+//xzbNiwAUFBQejcuTPeeustXL16Fampqbh8+TIAYO7cuSguLkZeXh5mz56N9957D5mZmdi3bx+CgoLQr18/JCcnIzc3F2+++SaCgoLQsmVLLFu27Oa3JhVXrlzB5cuX8fzzz2PcuHHu3zCRhygWoK5cuYLk5GRUVlaipqYGr732Gu677z6lLk9NhV4vHWUMBiA1FamFf0cVulh9yZxLIdcIWWxsLAYNGoQPP/wQAFBUVISysjJs3rwZFy9eREFBQaPn7N27FxMnTsSwYcOwe/duVFZWYv369ejfvz/GjRuHgoICzJkzB9u3b0ePHj2wYMECnDp1Cp988gmysrIQFBSEadOm4csvv8Thw4cxaNAgTJo0CV988QUqKipw+fJlDBs2DE8++SRKSkoQHx/PAEWqpliAevfdd9G/f39MnDgRJ0+eRFJSEj766COlLk9NXb1uUxHE56TkzKW4/fbbrT6/8847odfrMWvWLNy4cQPx8fE4cuQIVq1aBQCYNGkS5syZg/Xr12P79u3o2rUrYmJicPz4cXz77bf45JNPAAAVFRVW5z158iR69+6N4OBgAEC/fv3w66+/YvLkyVi3bh0mTJiAqKgo9OrVC+3atcOWLVvw+eefIyIiAjdu3JDvhok8QLEANXHiRISEmIpz1dbWolmzZqKPy8vLs/rcaDQ2OuZveI+e1y05GSE3x/S0KEJhgx4UAHToUI28vHyXzm80GlFSUoJr167hxo0bOHXqFGpqanDu3DlcvHgRn3zyCfLz85GYmIhLly7htddeQ0ZGBlJSUiznePvttzFs2DCMGTMGa9aswZYtWxAZGYm+fftiwIAB+O2333DgwAHk5eXh2rVryM83tfXQoUP497//jYCAAHzxxRd44oknsGHDBvTu3RtPP/003n//fbz99tuoqqpCx44dMWTIEBw7dgzZ2dlO/Uy8/TNUgr/fo6/dn0cC1K5du7BlyxarY4sWLUKvXr1QWlqK5ORkq1/M+nr06GH1eV5eXqNj/ob3qIDz5y3/TUcKErABVWhuORYeDixdGuJyG/Py8tC+fXuEhYWhWbNm6Nq1K7p164a8vDwYjUYMHDgQn3zyCebNm4fg4GAkJSU1utbAgQOxePFiREZGonnz5pbht9TUVHz99deorKzEq6++ih49euCRRx7BunXrsGnTJpw/fx4LFy5EXV0d+vbti4kTJ+Lo0aNYuHAhwsPDERwcjLS0NJw9exYLFizAd999h8jISISGhqJbt26WN46O3CNfp75NrfeXm5sr/gVBQT///LMwdOhQ4e9//7vo148cOdLo2H/+8x9PN8vreI8K0OkEwVRfQhAAIRNjBR1OCRrUCjqdIGRmund6r9+fAniPvk+t9yf2t18QBEGxNPMTJ05gxowZWL58OQYMGKDUZYlMGqSg67EdBeH3oC5zO5cPEamUYnNQy5cvR3V1NdJvrkmJiIjA2rVrlbo8NXUOpqATkXooFqAYjMjrbKWgE5HqsJIEERGpEgMUERGpEgMUERGpEgMUkUxkr2buhoyMDBw9etSp58THx1sW/xKpAQOUSvnLzuWqpdA32FPVzO1JSEhAr169FL8ukZxYzVyFPF1tu8nz8DfYlWrmNTU1GDp0KP72t78hPDwcGzduRFBQEJ566inMmzcP169fR7NmzfD666+jtrYWU6ZMQWRkJB577DGEh4dj9+7dCAgIQJ8+fTB79my89tprGDp0KB544AHMmTMH586dQ01NDebNm4f/+q//QkpKCk6fPo3a2lr88Y9/xNChQy1tqaiosBR2rq2txYwZM/DQQw9h+vTp6N69O0JCQvB///d/bn+fiOxhgFIhWzuXM0DJQIFvsLPVzIODg/Hkk0/i888/xzPPPIP9+/fjnXfewcKFCxEfH48BAwbgm2++wbJly5CYmIjS0lJ88MEHCAkJwXPPPYd58+bh3nvvxXvvvWdVBDYrKwudO3fGihUrcPz4cfzzn//ETz/9hNatW2Pp0qWorKzEs88+i/79+1ues3btWjz88MOYMGECSkpKMHbsWGRnZ+PatWuYOnUq7r77blm+R0T2MECpkD/tXK5KCnyDXalm/vzzz2PBggXo2rUrunTpgtatW+P48eNYv349Nm7cCEEQLFXLb7vtNksNvcWLF2PTpk1YtmwZ7r33XgiCYLnuyZMn8dhjjwEA7rrrLtx1111YuHAhHn74YQCmBfPdunXD6dOnLc/Jz89HbGwsACAqKgoRERG4dOmS6H0ReRIDlApptaZRJ7HjJAMFvsEajcbq819++QVXr15FRkYGLly4gLi4OHzxxRfYtm2b1eMEQcDGjRsxduxYAEDXrl3x4osvok+fPsjPz8d3330HAAgIuDV9vHPnTixcuBDNmjXDpEmT8P3331u+1q1bNxw7dgwxMTE4ffo0Vq5cifvuuw9HjhzBoEGDUFlZiePHj+O2226zes6RI0dw9913o6SkBBUVFYiMjGx0XSJPY4BSofR06ykSQHLncnKFF77BXbp0wdtvv43du3cjODgY06dPF33cqFGjsGrVKsuQ2+zZs7FgwQJcv34dRqMRqampjZ7TvXt3jBo1Cq1bt0ZUVBR69+5tGVqMi4tDSkoKXnjhBdTW1iIlJQXdu3fHvHnzMHbsWFy/fh2vvvoq2rZtaznfyy+/jJSUFHz22WcwGo1IS0tDUBD/VJAXKFmx1h5WM78lM9NUgFujEWSptu1tqvs5yvwNVt39eQDv0fep9f6kqpnzbZFKsWych/EbTKR6HFAmIiJVYoAiIiJVYoAiIiJVYoAiIiJVYoAiIiJVYoAissfBwrKeqGbuTFVye9f68MMPkZOT43JbiJTGNHNqkgzHDEjNSUVReRG0rbRIj06HvqdI2rmbhWXNZYZclWC+lgPsXevZZ591qy1ESmOAoibHcMyAhD0JqKoxBZ3C8kIk7DEFgkZByoXCsnJWM//5558xdOhQlJWV4YMPPkBdXR2mT5+OM2fOwGAwoFWrVggODrZUIz958iTi4uKQlJSEDh064PTp0+jZsycWLlyI1atXo127dhgzZgzeeOMNHD16FDU1NZg2bRqeeOIJzJ8/H+fPn8fly5fx2GOPYebMmTJ9x4lcwyE+anJSc1ItwcmsqqYKqTmNywi5Wlg2NjYWmzdvRmBg4M2Hm6qZr1u3DsuXL4fRaLR6fP1q5gCwf/9+jBgxwuoxLVu2xPbt29G9e3ds3LgR27dvx6ZNm3Dt2rVG1y8oKEB6ejp27dqFgwcPorS01PK1nJwcXL58Ge+//z42btyIY8eOobi4GPfeey/eeecdbN++Hdu3b7d5f0RKYA+KmpyicvHgInrcxcKyclUzFztnUVERunXrhrCwMADAfffdJ9I8LSIiIgAA7du3x/Xr1y1fO3XqFO69917L1xITE1FZWYljx47h22+/RUREBKqrq23eH5ESGKCoydG20qKwvHHQ0bYSCTouFpaVq5p5feZK4lqtFidPnoTRaERISAiOHj2Krl272rx+fV27dsWnn34KALhy5QpmzpyJAQMGoEWLFkhLS0NhYSF27twJQRBsnofI0xigqMlJj063moMCgPDgcKRHiwQd8zxTaqppWE+rNQUnJ+v4uVrNXEybNm3w0ksvYdy4cYiMjMT169cRFBRktVGhLdHR0fjmm28wduxY1NbW4pVXXkGnTp0wa9Ys5ObmIiwsDDqdDhcuXEBUVJRT90kkK0VL1trBaub+S233mHk0U9Ct0AmaBRpBt0InZB71nWrmNTU1wpo1ayyfjxs3Tjh8+LDHr6u2n6En+Ps9qvX+WM2cqB59T714WrkPCAoKwrVr1zBy5EgEBwejV69e6Nevn7ebRSQ7BigiHzRr1izMmjXL280g8iimmRMRkSoxQBERkSoxQBERkSoxQBERkSopliRRVVWFpKQklJeXIywsDEuXLkWbNm2UujwREfkYxXpQO3fuxD333IP33nsPw4YNw5o1a5S6NBER+SDFelATJ05EbW0tAODcuXNo166dUpcmIiIfpBEEQZD7pLt27cKWLVusji1atAi9evXC+PHjcfz4cbz77rvo0aOH1WNyc3PlbgoREfmAvn37NjrmkQBlT35+Pl5++WVkZ2crfWkiIvIRis1BrV+/Hrt37wYAhIeHW/bJISIiEqNYD6qsrAyzZ89GdXU1amtrkZSUJNqlIyIiArw0xEdERGSP6hfqVlVVYcqUKRg3bhwmTZqES5cuebtJsrty5QomT56MF154AWPGjMH333/v7SZ5xIEDB5CUlOTtZsiqrq4O8+fPx5gxYxAfH49Csd13/cCPP/6I+Ph4bzfDI2pqapCcnIxx48Zh1KhRyMnJ8XaTZFdbW4s5c+YgLi4Oer0eRUXiu0qrjeoDVFNYP/Xuu++if//+yMzMxOLFi5GWlubtJsnujTfewPLly1FXV+ftpsgqOzsb1dXV2LFjB5KSkrBkyRJvN0l2GzZswNy5c622jfcnH3/8MSIjI/Hee+9hw4YNeP31173dJNl9+eWXAICsrCxMnz4dixcv9nKLHKP67TaawvqpiRMnIiQkBIDpnU6zZs283CL59enTBzExMdixY4e3myKr3NxcPProowCAe++9F//+97+93CL5abVarF69Gn/605+83RSPGDx4MJ566inL5/6YwBUTE4PHH38cgG/9HVVVgHJ0/ZQvs3WPpaWlSE5ORkpKipda5z6p+xs6dCgOHTrkpVZ5TmVlJSIiIiyfBwYG4saNGwgKUtWvllueeuopnDlzxtvN8JjmzZsDMP0sp0+fjpkzZ3q5RZ4RFBSE2bNn48CBA/jLX/7i7eY4RqEdfWVx4sQJITo62tvN8Iiff/5ZGDp0qPD3v//d203xmG+//VaYOXOmt5shq0WLFgn79u2zfP7oo496sTWec/r0aeH555/3djM85ty5c8LIkSOFXbt2ebspHnfhwgXh8ccfF65evertptil+jmoprB+6sSJE5gxYwaWL1+OAQMGeLs55IQ+ffrg4MGDAIAffvgBd911l5dbRM4qKyvDiy++iOTkZIwaNcrbzfGI3bt3Y/369QCAsLAwaDQan/hbqvpxiOeeew6zZ8/GBx98gNraWixatMjbTZLd8uXLUV1djfT0dABAREQE1q5d6+VWkSMGDRqEr7/+GnFxcRAEwS9fn/5u3bp1qKiowJo1ayxJWBs2bEBoaKiXWyafJ598EnPmzIFer8eNGzeQkpLiE3PdXAdFRESqpPohPiIiapoYoIiISJUYoIiISJUYoIiISJUYoIiISJUYoMinHTp0CA899BDi4+MRHx+P0aNHY9u2bY0ed/DgQafLLH344YdOFw49c+YMRo8e3eh4eXk5UlJSoNfrERcXh8TERFy5csWpc3vbjh07UFNTI/o1fywETN6n+nVQRPb0798fK1asAABUV1dj8ODBGDFiBFq2bGl5zGOPPeb0eZ999lnZ2jhr1izExcVh0KBBAIDNmzdj/vz5lnb7gvXr1+OZZ55pdPyNN97AV199hR49enihVeTPGKDIr1RWViIgIACBgYGIj49H69atUVFRgWHDhqGwsBBxcXFISkpChw4dcPr0afTs2RMLFy7ExYsX8dprr+HKlSsQBAFvvvkm9uzZg3bt2qFr165Yt24dAgICUFpaijFjxkCv1+Pw4cP461//CgAwGo148803ERwc3KhNZ8+eRVlZmSU4AUB8fDyee+45AKZq2lu2bEFISAi6dOmCtLQ07NmzB19++SWMRiNKS0sxfvx45OTk4Ndff8Wf/vQnxMTEIDo6Gr1790ZRURHuvPNOpKeno7KyEsnJyaisrERtbS1mzJiBhx56CLGxsXjggQfwyy+/QKPRYM2aNWjRogWWL1+O7777DoIgYOLEiRgyZAji4+Pxhz/8Ab/++isqKyuxatUq/POf/0RpaSkSExMb7Sjgr4WAyfsYoMjnffvtt4iPj4dGo0FwcDDmzZtnKQAaGxuLQYMG4cMPP7Q8vqCgAO+88w7CwsIQExOD0tJSrF+/HgMHDsTYsWPxzTff4OjRo1bXKCkpwe7du1FXV4fY2FgMHjwYv/76K5YuXYqoqCisW7cOn376KWJjYxu178KFC7jtttusjgUGBqJFixa4fPkyVq9ejY8++ggRERFYtGgRduzYgfDwcFy9ehWbNm3Cvn37sHnzZuzcuROHDh3C1q1bERMTg5KSEsyYMQM6nQ4zZsxAdnY2vv/+ezz88MOYMGECSkpKMHbsWGRnZ+Pq1asYNmwY5s2bh6SkJBw8eBARERE4c+YMsrKycP36dYwePRqPPPIIAKBXr15ITU3FihUrsG/fPiQkJGDt2rWiPT5/LQRM3scART6v/hBfQ7fffnujY1qt1lKBvH379rh+/TpOnTplqcP20EMPAQBWr15tec59991n2RLlzjvvRFFREaKiopCeno7w8HCUlJSgT58+om3o1KkTzp8/b3WspqYGn376KXQ6He644w5Le+6//3589dVX6N27t2XIrEWLFujWrRs0Gg1atWpl2ZepY8eO0Ol0lvadOnUK+fn5liAZFRWFiIgIyyafd999t+V5169fx7lz5/DTTz9ZNiK8ceMGzp07Z/XYDh06oKysTPS+iDyNSRLk1zQajUPHunXrhmPHjgEAvvvuOyxdutTq63l5eaitrcW1a9dw4sQJ6HQ6zJ07F4sWLcKSJUvwu9/9DlJVw6KiotC6dWtkZ2dbjm3duhXZ2dm47bbbkJ+fj6qqKgDA4cOHLUFVrJ31lZSUoLS0FADwr3/9C3fccQe6deuGI0eOWL5eUVGByMhI0fN17doVDz74ILZt24YtW7ZgyJAhjXp69Wk0Gr/bcJLUjT0oIgCTJ09GSkoKPv74YwCmPazMVfQBU+/ipZdewm+//YYpU6agTZs2GDFiBEaPHo2WLVuiXbt2uHDhguT533rrLaSlpWHTpk2oqamBVqvFG2+8gRYtWmDatGkYP348AgICoNVq8b//+7/Yt2+f3TaHhITg9ddfR3FxMXr37o2BAweib9++SElJwWeffQaj0Yi0tDTJvakGDhyIw4cPY9y4caiqqkJMTIzV3lYN9evXDwkJCdi6davd4EkkBxaLJbLj0KFDyMrKUl3G3SOPPIKvv/7a280g8hgO8RERkSqxB0VERKrEHhQREakSAxQREakSAxQREakSAxQREakSAxQREanS/wdomkT0qRET9QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sklearn_pca = sklearnPCA(n_components=2)\n",
    "Y_sklearn = sklearn_pca.fit_transform(X_std)\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(Y_sklearn[y==lab, 0],\n",
    "                    Y_sklearn[y==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "- [PCA Jupyter Notebook by Sebastian Raschka](https://nbviewer.jupyter.org/github/rasbt/pattern_classification/blob/master/dimensionality_reduction/projection/principal_component_analysis.#PCA-and-Dimensionality-Reduction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
